Information processing apparatus and method, and projection imaging apparatus and information processing method

ABSTRACT

There is provided an information processing apparatus and method, and a projection imaging apparatus and an information processing method capable of suppressing an increase in processing amount of processing related to update of correction information used for geometric correction and the like, the information processing apparatus detecting, in regard to projection imaging apparatuses each having a projection unit that projects an image on a projection surface and an imaging unit that images the projection surface, a corresponding point between a pixel of the projection unit and a pixel of the imaging unit for a relation between one of the projection imaging apparatuses whose posture has changed and one of the projection imaging apparatuses whose posture has not changed, and estimating a relative posture of the projection imaging apparatus whose posture has changed with respect to the projection imaging apparatus whose posture has not changed.

CROSS REFERENCE TO PRIOR APPLICATION

This application is a National Stage Patent Application of PCTInternational Patent Application No. PCT/JP2016/068753 (filed on Jun.24, 2016) under 35 U.S.C. § 371, which claims priority to JapanesePatent Application No. 2015-136727 (filed on Jul. 8, 2015), which areall hereby incorporated by reference in their entirety.

TECHNICAL FIELD

The present technology relates to an information processing apparatusand method, and a projection imaging apparatus and an informationprocessing method, and more particularly, to an information processingapparatus and method, and a projection imaging apparatus and aninformation processing method capable of suppressing an increase inprocessing amount of processing related to update of correctioninformation used for geometric correction and the like.

BACKGROUND ART

Conventionally, there has been a technology such as so-called projectionmapping or the like in which images are projected by causing a pluralityof projectors to collaborate. For example, there has been a method ofprojecting images using a plurality of projectors to form a singleprojection image as a whole. In such a case, in order to appropriatelycombine the projection images projected by the respective projectorswithout giving any sense of incompatibility and form one projectionimage, correction processing such as position alignment and geometriccorrection have been performed on each image to be projected. How tocorrect has been set according to the posture of the projector (position(translation component) and direction (rotation component)), the shapeof a projection surface, and so on.

There has been a method of estimating the posture of the projector andthe shape of the projection surface using a camera that captures aprojection image projected by the projector to obtain a captured image.Estimation of the shape of the projection surface and estimation of theposture of each projector have been enabled using the projected imageand the captured image on the basis of a correspondence relationship ofpixels between the projector and the camera and a relative positionalrelationship between the projector and the camera.

However, there has been a possibility that the posture of the projectorand the shape of the projection surface change even during theprojection of an image such as contents. Then, in a case where such achange occurs, it has been necessary to re-estimate these elements andto set up again how to correct (to update setting information indicatinghow to set these elements). That is, it has been necessary to find thecorrespondence relationship of pixels between the projector and thecamera and to estimate the posture of the projector and the shape of theprojection surface using this correspondence relationship.

For example, as online sensing which is a technology for finding thecorrespondence relationship of pixels while an image such as contents isbeing projected, a technique of embedding a gray code in a projectionimage (for example, refer to Non-Patent Document 1), a techniqueutilizing an image feature amount such as SIFT, and a techniqueutilizing invisible light such as infrared light (Infrared) have beenproposed.

CITATION LIST Non-Patent Document

-   Non-Patent Document 1: Imperceptible Structured Light Ramesh Raskar,    SIGGRAPH 98

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

However, in any case of these methods, correction information has beenupdated for all the projectors. That is, as described above, all itemsof processing such as the calculation of a pixel correspondencerelationship, the posture estimation for the projector, and the shapeestimation for the projection surface have been performed for all theprojectors. For example, even in a case where the postures of someprojectors have changed, all of these items of the processing have beenperformed for all the projectors. Therefore, there has been a risk thatthe processing amount of processing related to the update of thecorrection information used for geometric correction and the likeincreases and the processing time increases.

The present technology has been proposed by taking such a situation inconsideration and an object thereof is to suppress an increase inprocessing amount of processing related to update of correctioninformation used for geometric correction and the like.

Solutions to Problems

An information processing apparatus according to the present technologyis an information processing apparatus including a corresponding pointdetection unit that detects, in regard to projection imaging apparatuseseach having a projection unit that projects an image on a projectionsurface and an imaging unit that images the projection surface, acorresponding point between a pixel of the projection unit and a pixelof the imaging unit for a relation between one of the projection imagingapparatuses whose posture has changed and one of the projection imagingapparatuses whose posture has not changed; and a relative postureestimation unit that estimates a relative posture of the projectionimaging apparatus whose posture has changed with respect to theprojection imaging apparatus whose posture has not changed on the basisof the corresponding point between a pixel of the projection unit and apixel of the imaging unit detected by the corresponding point detectionunit for a relation between the projection imaging apparatus whoseposture has changed and the projection imaging apparatus whose posturehas not changed.

The corresponding point detection unit can detect a corresponding pointbetween a pixel of the projection unit of the projection imagingapparatus whose posture has not changed and a pixel of the imaging unitof the projection imaging apparatus whose posture has changed.

The corresponding point detection unit can detect a corresponding pointbetween a pixel of the projection unit of the projection imagingapparatus whose posture has changed and a pixel of the imaging unit ofthe projection imaging apparatus whose posture has not changed.

A corresponding point change detection unit that detects a change in acorresponding point between a pixel of the projection unit and a pixelof the imaging unit of the projection imaging apparatuses; a posturechange detection unit that detects a posture change in each projectionimaging apparatus on the basis of a sensor output from a sensor unitwhich is included in each projection imaging apparatus and detects atleast one of a position and a direction; and a change determination unitthat determines whether a posture of each projection imaging apparatushas changed on the basis of a detection result of the correspondingpoint change detection unit and a detection result of the posture changedetection unit can be further included, in which the corresponding pointdetection unit can be configured to detect the corresponding pointbetween one of the projection imaging apparatuses whose posture isdetermined by the change determination unit to have changed and one ofthe projection imaging apparatuses whose posture is determined by thechange determination unit to have not changed, and the relative postureestimation unit can be configured to estimate a relative posture of theprojection imaging apparatus whose posture is determined by the changedetermination unit to have changed with respect to the projectionimaging apparatus whose posture is determined by the changedetermination unit to have not changed.

The corresponding point detection unit can be configured to furtherdetect a corresponding point between a pixel of the projection unit anda pixel of the imaging unit of the projection imaging apparatus whoseposture has changed, and a projection surface shape estimation unit thatestimates a shape of the projection surface on the basis of thecorresponding point detected by the corresponding point detection unitfor a relation between a pixel of the projection unit and a pixel of theimaging unit of the projection imaging apparatus whose posture haschanged can be further included.

The corresponding point detection unit can be configured to furtherdetect a corresponding point between a pixel of the projection unit anda pixel of the imaging unit of the projection imaging apparatus whoseposture has not changed, and a projection surface shape estimation unitthat estimates a shape of the projection surface on the basis of thecorresponding point detected by the corresponding point detection unitfor a relation between a pixel of the projection unit and a pixel of theimaging unit of the projection imaging apparatus whose posture has notchanged can be further included.

The projection surface shape estimation unit can estimate a shape of theprojection surface for a part of the projection surface whose shape isunknown within a range on which an image is projected by the projectionunit of the projection imaging apparatus whose posture has changed.

An information processing method according to the present technology isan information processing method configured to detect, in regard toprojection imaging apparatuses each having a projection unit thatprojects an image on a projection surface and an imaging unit thatimages the projection surface, a corresponding point between a pixel ofthe projection unit and a pixel of the imaging unit for a relationbetween one of the projection imaging apparatuses whose posture haschanged and one of the projection imaging apparatuses whose posture hasnot changed; and estimate a relative posture of the projection imagingapparatus whose posture has changed with respect to the projectionimaging apparatus whose posture has not changed on the basis of thecorresponding point between a pixel of the projection unit and a pixelof the imaging unit detected for a relation between the projectionimaging apparatus whose posture has changed and the projection imagingapparatus whose posture has not changed.

In addition, an information processing apparatus according to thepresent technology is an information processing apparatus including acorresponding point detection unit that detects, in regard to aprojection imaging apparatus having a projection unit that projects animage on a projection surface and an imaging unit that images theprojection surface, a corresponding point between a pixel of theprojection unit and a pixel of the imaging unit of the projectionimaging apparatus whose posture has not changed; and a projectionsurface shape estimation unit that estimates a shape of the projectionsurface on the basis of the corresponding point detected by thecorresponding point detection unit for a relation between a pixel of theprojection unit and a pixel of the imaging unit of the projectionimaging apparatus whose posture has not changed.

The projection surface shape estimation unit can estimate a shape of theprojection surface for a part of the projection surface whose shape isunknown.

In addition, an information processing method according to the presenttechnology is an information processing method configured to detect, inregard to a projection imaging apparatus having a projection unit thatprojects an image on a projection surface and an imaging unit thatimages the projection surface, a corresponding point between a pixel ofthe projection unit and a pixel of the imaging unit of the projectionimaging apparatus whose posture has not changed; and estimate a shape ofthe projection surface on the basis of the corresponding point detectedfor a relation between a pixel of the projection unit and a pixel of theimaging unit of the projection imaging apparatus whose posture has notchanged.

A projection imaging apparatus according to the present technology is aprojection imaging apparatus including a projection unit that projectsan image on a projection surface; an imaging unit that images theprojection surface; a sensor unit that detects at least one of aposition and a direction; a determination unit that determines whether aposture has changed on the basis of a sensor output from the sensorunit; a corresponding point detection unit that detects, in a case wherethe determination unit determines that the posture has changed, acorresponding point between a pixel of the projection unit and a pixelof the imaging unit for a relation with another projection imagingapparatus which includes the projection unit, the imaging unit, and thesensor unit and whose posture has not changed; and a relative postureestimation unit that estimates a relative posture with respect to theanother projection imaging apparatus on the basis of the correspondingpoint detected by the corresponding point detection unit.

The corresponding point detection unit can detect a corresponding pointbetween a pixel of the projection unit of the another projection imagingapparatus and a pixel of the own imaging unit of the projection imagingapparatus.

The corresponding point detection unit can detect a corresponding pointbetween a pixel of the own projection unit of the projection imagingapparatus and a pixel of the imaging unit of the another projectionimaging apparatus.

A corresponding point change detection unit that detects a change in acorresponding point between a pixel of the own projection unit and apixel of the own imaging unit of the projection imaging apparatus; and aposture change detection unit that detects an own posture change in theprojection imaging apparatus on the basis of a sensor output from thesensor unit can be further included, in which the determination unit candetermine whether an own posture of the projection imaging apparatus haschanged on the basis of a detection result of the corresponding pointchange detection unit and a detection result of the posture changedetection unit.

The corresponding point detection unit can be configured to furtherdetect a corresponding point between a pixel of the own projection unitand a pixel of the own imaging unit of the projection imaging apparatus,and a projection surface shape estimation unit that estimates a shape ofthe projection surface on the basis of the corresponding point detectedby the corresponding point detection unit for a relation between a pixelof the own projection unit and a pixel of the own imaging unit of theprojection imaging apparatus can be further included.

The projection surface shape estimation unit can estimate a shape of theprojection surface for a part of the projection surface whose shape isunknown within a range on which an image is projected by the ownprojection unit of the projection imaging apparatus.

Furthermore, an information processing method according to the presenttechnology is an information processing method for a projection imagingapparatus including a projection unit that projects an image on aprojection surface; an imaging unit that images the projection surface;and a sensor unit that detects at least one of a position and adirection, the information processing method being configured to:determine whether a posture has changed on the basis of a sensor outputfrom the sensor unit; detect, in a case where the posture is determinedto have changed, a corresponding point between a pixel of the projectionunit and a pixel of the imaging unit for a relation with anotherprojection imaging apparatus which includes the projection unit, theimaging unit, and the sensor unit and whose posture has not changed; andestimate a relative posture with respect to the another projectionimaging apparatus on the basis of the detected corresponding point.

In addition, a projection imaging apparatus according to the presenttechnology is a projection imaging apparatus including a projection unitthat projects an image on a projection surface; an imaging unit thatimages the projection surface; a sensor unit that detects at least oneof a position and a direction; a determination unit that determineswhether a posture has changed on the basis of a sensor output from thesensor unit; a corresponding point detection unit that detects, in acase where the determination unit determines that the posture has notchanged, a corresponding point between a pixel of the own projectionunit and a pixel of the own imaging unit of the projection imagingapparatus; and a projection surface shape estimation unit that estimatesa shape of the projection surface on the basis of the correspondingpoint detected by the corresponding point detection unit.

In addition, an information processing method according to the presenttechnology Is an information processing method for a projection imagingapparatus including a projection unit that projects an image on aprojection surface; an imaging unit that images the projection surface;and a sensor unit that detects at least one of a position and adirection, the information processing method being configured to:determine whether a posture has changed on the basis of a sensor outputfrom the sensor unit; detect, in a case where the posture is determinedto have not changed, a corresponding point between a pixel of the ownprojection unit and a pixel of the own imaging unit of the projectionimaging apparatus; and estimate a shape of the projection surface on thebasis of the detected corresponding point.

In the information processing apparatus and method according to thepresent technology, in regard to projection imaging apparatuses eachhaving a projection unit that projects an image on a projection surfaceand an imaging unit that images the projection surface, a correspondingpoint between a pixel of the projection unit and a pixel of the imagingunit is detected for a relation between one of the projection imagingapparatuses whose posture has changed and one of the projection imagingapparatuses whose posture has not changed, and a relative posture of theprojection imaging apparatus whose posture has changed with respect tothe projection imaging apparatus whose posture has not changed isestimated on the basis of the corresponding point between a pixel of theprojection unit and a pixel of the imaging unit detected for a relationbetween the projection imaging apparatus whose posture has changed andthe projection imaging apparatus whose posture has not changed.

In another information processing apparatus and method according to thepresent technology, in regard to a projection imaging apparatus having aprojection unit that projects an image on a projection surface and animaging unit that images the projection surface, a corresponding pointbetween a pixel of the projection unit and a pixel of the imaging unitof the projection imaging apparatus whose posture has not changed isdetected, and a shape of the projection surface is estimated on thebasis of the corresponding point detected for a relation between a pixelof the projection unit and a pixel of the imaging unit of the projectionimaging apparatus whose posture has not changed.

In the projection imaging apparatus and the information processingmethod according to the present technology, in a projection imagingapparatus including a projection unit that projects an image on aprojection surface, an imaging unit that images the projection surface,and a sensor unit that detects at least one of a position and adirection, whether a posture has changed is determined on the basis of asensor output from the sensor unit, a corresponding point between apixel of the projection unit and a pixel of the imaging unit is detectedfor a relation with another projection imaging apparatus which includesthe projection unit, the imaging unit, and the sensor unit and whoseposture has not changed in a case where the posture is determined tohave changed, and a relative posture with respect to the anotherprojection imaging apparatus is estimated on the basis of the detectedcorresponding point.

In another projection imaging apparatus and another informationprocessing method according to the present technology, in a projectionimaging apparatus including a projection unit that projects an image ona projection surface, an imaging unit that images the projectionsurface, and a sensor unit that detects at least one of a position and adirection, whether a posture has changed is determined on the basis of asensor output from the sensor unit, a corresponding point between apixel of the own projection unit and a pixel of the own imaging unit ofthe projection imaging apparatus is detected in a case where the postureis determined to have not changed, and a shape of the projection surfaceis estimated on the basis of the detected corresponding point.

Effects of the Invention

According to the present technology, information can be processed.Additionally, according to the present technology, it is possible tosuppress an increase in processing amount of processing related toupdate of correction information used for geometric correction and thelike.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram for explaining a main configuration example of aprojection imaging system.

FIG. 2 is a diagram for explaining an example of how a posture changes.

FIG. 3 is a diagram for explaining an example of how the shape of aprojection surface changes.

FIG. 4 is a diagram for explaining an example of how a calibrationproceeds.

FIG. 5 is a diagram for explaining an example of how a calibrationproceeds.

FIG. 6 is a diagram for explaining an example of a method using a graycode.

FIG. 7 is a diagram for explaining an example of processing executed asa calibration.

FIG. 8 is a diagram for explaining an example of how a posture isestimated.

FIG. 9 is a diagram for explaining an example of how the shape of aprojection surface is estimated.

FIG. 10 is a block diagram illustrating a main configuration example ofa control apparatus.

FIG. 11 is a view for explaining an example of the appearance of aprojection imaging apparatus.

FIG. 12 is a block diagram illustrating a main configuration example ofthe projection imaging apparatus.

FIG. 13 is a block diagram illustrating a main configuration example ofa projection unit.

FIG. 14 is a diagram illustrating an example of scanning of a laserbeam.

FIG. 15 is a flowchart for explaining an example of a flow of correctioninformation update processing.

FIG. 16 is a flowchart for explaining an example of a flow ofcorresponding point detection processing.

FIG. 17 is a flowchart for explaining an example of a flow of projectionsurface shape estimation processing.

FIG. 18 is a flowchart for explaining an example of a flow ofcorresponding point detection processing.

FIG. 19 is a flowchart for explaining an example of a flow of projectionsurface shape estimation processing.

FIG. 20 is a flowchart for explaining an example of a flow ofcorresponding point detection processing.

FIG. 21 is a diagram illustrating another configuration example of theprojection imaging system.

MODE FOR CARRYING OUT THE INVENTION

Modes for carrying out the present disclosure (hereinafter, referred toas embodiments) will be described below. Note that the description willbe given in the following order.

1. First Embodiment (Projection Imaging System)

2. Second Embodiment (Corresponding Point Detection Processing)

3. Third Embodiment (Projection imaging System)

1. First Embodiment

<Projection Imaging System>

FIG. 1 illustrates a main configuration example of a projection imagingsystem to which a control apparatus which is an embodiment of aninformation processing apparatus to which the present technology isapplied and a projection imaging apparatus which is an embodiment of aprojection imaging apparatus to which the present technology is appliedare applied. The projection imaging system 100 illustrated in FIG. 1 isa system that projects an image using a plurality of projection imagingapparatuses 102.

As illustrated in FIG. 1, the projection imaging system 100 has acontrol apparatus 101, a projection imaging apparatus 102-1 to aprojection imaging apparatus 102-4, and a communication cable 103-1 to acommunication cable 103-4.

The projection imaging apparatuses 102-1 to 102-4 are connected to thecontrol apparatus 101 so as to be able to communicate therewith via thecommunication cables 103-1 to 103-4, respectively.

The control apparatus 101 communicates with each of the projectionimaging apparatuses 102-1 to 102-4 and controls the actions thereof. Forexample, the control apparatus 101 causes the projection imagingapparatuses 102-1 to 102-4 to project an image on a projection surface104 and to capture an image (projection image) projected on theprojection surface 104.

In addition, for example, the control apparatus 101 can calibraterelative postures (for example, rotation components (relativedirections) and translation components (relative positions)) of aprojection unit 111 and an imaging unit 112 included in each of theprojection imaging apparatuses 102-1 to 102-4. Furthermore, for example,the control apparatus 101 can make correction (position alignment,geometric correction, and so on) on an image projected by each of theprojection imaging apparatuses 102-1 to 102-4 using a result of thecalibration.

Each of the projection imaging apparatuses 102-1 to 102-4 is controlledby the control apparatus 101 to project an image on the projectionsurface 104 and to image the projection surface 104 (for example, aprojection image projected on the projection surface 104). That is, theprojection imaging apparatuses 102-1 to 102-4 have similarconfigurations to each other to have similar functions. In a case whereit is not necessary to distinguish these projection imaging apparatuses102-1 to 102-4 from each other when described, these projection imagingapparatuses are referred to as projection imaging apparatuses 102.

The projection imaging apparatus 102 has the projection unit 111 and theimaging unit 112. That is, the projection imaging apparatus 102-1 has aprojection unit 111-1 and an imaging unit 112-1. The projection imagingapparatus 102-2 has a projection unit 111-2 and an imaging unit 112-2.The projection imaging apparatus 102-3 has a projection unit 111-3 andan imaging unit 112-3. The projection imaging apparatus 102-4 has aprojection unit 111-4 and an imaging unit 112-4.

Each of the projection units 111-1 to 111-4 projects an image. That is,the projection units 111-1 to 111-4 have similar configurations to eachother to have similar functions. In a case where it is not necessary todistinguish these projection units 111-1 to 111-4 from each other whendescribed, these projection units are referred to as projection units111. In addition, each of the imaging units 112-1 to 112-4 images asubject to obtain a captured image. That is, the imaging units 112-1 to112-4 have similar configurations to each other to have similarfunctions. In a case where it is not necessary to distinguish theseimaging units 112-1 to 112-4 from each other when described, theseimaging units are referred to as imaging units 112.

Meanwhile, each of the communication cables 103-1 to 103-4 is acommunication medium supporting a predetermined communication standard.That is, the communication cables 103-1 to 103-4 have similarconfigurations to each other to have similar functions. In a case whereit is not necessary to distinguish these communication cables 103-1 to103-4 from each other when described, these communication cables arereferred to as communication cables 103.

The communication cable 103 is, for example, a cable conforming tohigh-definition multimedia interface (HDMI) (registered trademark). As amatter of course, the communication standard support by thecommunication cable 103 is arbitrary and the communication cable 103 maysupport a communication standard other than the HDMI (registeredtrademark), such as a display port (DisplayPort).

The projection surface 104 is an example of a destination on which theprojection imaging apparatus 102 projects an image (projectiondestination). The projection surface 104 may be a flat surface, a curvedsurface, or a surface having irregularities in part or the whole, or maybe made up of a plurality of surfaces. In addition, the color of theprojection surface 104 is arbitrary and the projection surface 104 maybe made up of a plurality of colors or may have a pattern or a picture.

Incidentally, this projection surface 104 may be formed on an arbitraryobject. For example, the projection surface 104 may be formed on aplanar object such as a so-called screen or a wall surface. Theprojection surface 104 also may be formed on a three-dimensionalstructure. For example, the projection surface 104 may be formed on wallsurfaces of buildings, station buildings, castles and other constructsor may be formed on, for example, natural objects such as rocks,artificial objects such as signboards and statues, furniture such aschests of drawers, chairs, and desks. Alternatively, the projectionsurface 104 may be formed on organisms such as persons, animals orplants. The projection surface 104 also may be formed on a plurality ofsurfaces, such as a wall, a floor, a ceiling, and so on of a room space.

In addition, the projection surface 104 may be formed on a solid or maybe formed on a liquid or a gas. For example, the projection surface 104may be formed on a water surface of a pond, a pool, or the like, aflowing water surface of a waterfall, a fountain, or the like, or a gassuch as fog or mist. Meanwhile, the projection surface 104 may move ormay be deformed or discolored. In addition, the projection surface 104may be formed on a plurality of objects, for example, walls, furniture,and people in the room, a plurality of buildings, or castle walls andfountains.

<Projection Area>

In FIG. 1, a projection image 105-1 on the projection surface 104 is animage projected by the projection unit 111-1 of the projection imagingapparatus 102-1. Likewise, a projection image 105-2 is an imageprojected by the projection unit 111-2 of the projection imagingapparatus 102-2. A projection image 105-3 is an image projected by theprojection unit 111-3 of the projection imaging apparatus 102-3. Aprojection image 105-4 is an image projected by the projection unit111-4 of the projection imaging apparatus 102-4. In a case where it isnot necessary to distinguish these projection images 105-1 to 105-4 fromeach other when described, these projection images will be referred toas projection images 105.

Note that the positions and the shapes of the respective projectionimages 105 on the projection surface 104 are arbitrary but, in the caseof the example in FIG. 1, one projection area (an area where an image isprojected) is formed by the respective projection images 105 on thisprojection surface 104. The projection image 105-1 forms an upper leftpart of the one projection area, the projection image 105-2 forms alower left part of the one projection area, the projection image 105-3forms an upper right part of the one projection area, and the projectionimage 105-4 forms a lower right part of the one projection area. Inaddition, each projection image 105 is designed in such a manner that apart thereof covers (overlaps) a part of other projection image. Byproviding such an area (overlapping area) where a plurality ofprojection images cover each other, one projection area can be morereliably (more stably) formed by the plurality of projection images.Each projection imaging apparatus 102 (projection unit 111) is arrangedin such a posture (position and projection direction) as to realize suchone projection area.

<Projection Mapping>

The control apparatus 101 can perform so-called projection mapping bycontrolling each projection imaging apparatus 102. For example, thecontrol apparatus 101 can cause an image to be projected from eachprojection imaging apparatus 102 and to be projected as one projectionimage in the above-described one projection area.

For example, the control apparatus 101 can cause the respectiveprojection imaging apparatuses 102 to collaborate to project one inputimage 106 on the projection surface 104 as one corrected projectionimage 107. In the case of the example in FIG. 1, the control apparatus101 divides the one input image 106 into two in an up-down direction andinto two in a left-right direction (totally divides the one input image106 in four) and supplies each partial image to each projection imagingapparatus 102 to project on the projection surface 104. Then, theserespective projection images are combined on the projection surface 104to form one projection image. By controlling in such a manner, forexample, a projection image with a resolution higher than the resolutionof one projection unit 111 (in other words, a projection image with asize larger than the maximum size of an image projected by oneprojection unit 111) can be realized.

<Image Correction>

When the projection images are combined in this manner, in order toappropriately combine the projection images projected by the respectiveprojection units 111 without giving any sense of incompatibility suchthat one projection image (corrected projection image 107) is formed,the control apparatus 101 performs correction processing such asposition alignment, geometric correction, and image quality correction(for example, luminance, color, and resolution) on each image projectedby each projection imaging apparatus 102. Particularly, blendingprocessing or the like is performed on the overlapping area whereprojection is performed from the plurality of projection units 111 suchthat an image without sense of incompatibility is obtained.

<Setting of Correction Information>

How to correct the image during these corrections is set according tothe posture (position (translation component) and direction (rotationcomponent)) of each projection imaging apparatus 102 (projection unit111), the shape of the projection surface 104, and so on. That is, thecontrol apparatus 101 estimates the posture of each projection imagingapparatus 102 (projection unit 111) and the shape of the projectionsurface 104 and, on the basis of a result of the estimation, sets how tocorrect each image (sets setting information indicating how to set).

In addition, the control apparatus 101 finds a correspondencerelationship of pixels between the projection unit 111 and the imagingunit 112 and utilizes this correspondence relationship to estimate theposture of each projection imaging apparatus 102 (projection unit 111)and the shape of the projection surface 104. The relative posturebetween the projection imaging apparatuses 102 is estimated by patternprojection (structured light) from the projection units 111, imaging ofthe projection surface 104 by the imaging units 112, and so on carriedout among the plurality of projection imaging apparatuses 102.Meanwhile, shape estimation (depth sensing) for the projection surface104 is performed by the pattern projection (structured light) from theprojection unit 111, imaging of the projection surface 104 by theimaging unit 112, and so on carried out in each projection imagingapparatus 102.

<Initial Setting>

In the case of performing the projection mapping as described above, thecontrol apparatus 101 finds the correspondence relationship of pixelsbetween the projection unit 111 and the imaging unit 112 for all theprojection imaging apparatuses 102 prior to projecting an image such ascontents and estimates the posture of the projection imaging apparatus102 (projection unit 111) and the shape of the projection surface 104 toset the correction information. Then, when projecting an image such ascontents, the control apparatus 101 corrects the image in accordancewith this correction information.

Here, each projection imaging apparatus 102 is calibrated in advance.For example, the characteristics related to the projection by theprojection unit 111 and the characteristics related to the imaging bythe imaging unit 112 in each projection imaging apparatus 102 are clear(known). In addition, the posture (position and projection direction) ofthe projection unit 111 and the posture (position and imaging direction)of the imaging unit 112 in each projection imaging apparatus 102 arefixed and known. That is, a relationship of postures (also referred toas relative posture) between the projection unit 111 and the imagingunit 112 within each projection imaging apparatus 102 is known. Notethat the “projection direction” here indicates a direction in which theprojection unit 111 projects an image. Similarly, the “imagingdirection” indicates a direction in which the imaging unit 112 images asubject. Such internal variables and external variables of eachprojection imaging apparatus 102 (the projection unit 111 and theimaging unit 112) are calibrated using a dedicated apparatus or the likein advance (calibration, estimation of various variables, and so on).

Note that, when an image such as contents is projected, the relativeposture between the projection imaging apparatuses 102 is clear (known)because of estimation processing described above. That is, the relativeposture between the projection units 111, the relative posture betweenthe imaging units 112, and the relative posture between the projectionunit 111 and the imaging unit 112 are clear among the respectiveprojection imaging apparatuses 102.

In addition, the position and the shape of the projection surface 104(relative position and shape from each projection unit 111) are alsoclear (known). That is, the position and the shape of the projectionimage 105 by each projection imaging apparatus 102 (projection unit 111)on the projection surface 104 are also clear.

As described above, the control apparatus 101 causes the plurality ofprojection imaging apparatuses 102 to collaborate, thereby being able toenhance the resolution of the projection image (in other words, increasethe image size while suppressing a reduction in resolution (imagequality)). Note that the input image 106 (corrected projection image107) may be a moving image or may be a still image.

<Position Misalignment/Projection Surface Change>

As described above, the corrected projection image 107 is realized bycorrecting an image to be projected in accordance with the correctioninformation set in advance. Therefore, if an environment changes duringimage projection, there has been a possibility that the correctioninformation results in an inappropriate value and the image quality ofthe corrected projection image 107 is reduced.

For example, as illustrated in FIG. 2, when the projection imagingapparatus 102-3 moves (position misalignment occurs) while therespective projection imaging apparatuses 102 project images on theprojection surface 104 to project the corrected projection image 107,there has been a possibility that the relative posture and so on betweenthe projection imaging apparatus 102-3 and the projection surface 104change. Therefore, there has been a possibility of a change in the state(position, dimension, shape, image quality, and so on) of the projectionimage 105-3. That is, the state (position, dimension, shape, imagequality, and so on) of a partial image 107-3 of the corrected projectionimage 107 included in this projection image 105-3 can also change.Therefore, there has been a possibility that the image quality isreduced even in the corrected projection image 107, for example,distortion, partial change in image quality, or the like occurs or theimage is separated into a plurality of images not to be organized as oneprojection image.

In addition, for example, as illustrated in FIG. 3, when athree-dimensional object 121 is newly installed in front of a positionon the projection surface 104 where the projection by the projectionimaging apparatus 102-2 is performed while the respective projectionimaging apparatuses 102 project images on this projection surface 104 toproject the corrected projection image 107, at least a part of theprojection image 105-2 projected by the projection imaging apparatus102-2 is caused to be projected on the three-dimensional object 121,causing a possibility of the occurrence of a state in which the shape ofthe projection surface 104 changes as seen from the projection imagingapparatus 102-2. Therefore, there has been a possibility of a change inthe state (position, dimension, shape, image quality, and so on) of theprojection image 105-2. That is, the state (position, dimension, shape,image quality, and so on) of a partial image 107-2 of the correctedprojection image 107 included in this projection image 105-2 can alsochange. Therefore, there has been a possibility that the image qualityis reduced even in the corrected projection image 107, for example,distortion, partial change in image quality, or the like occurs or theimage is separated into a plurality of images not to be organized as oneprojection image.

Therefore, when the position misalignment of the projection imagingapparatus 102, the shape change in the projection surface 104, or thelike occurs during the image projection as described above, thecorrection information for geometric correction or the like needs to beset again in order to suppress a reduction in image quality of theprojection image.

<Overlap Estimation>

As described above, when one video is projected using the plurality ofprojection imaging apparatuses 102, a portion where the projection bythe respective projection units 111 is superimposed on the projectionsurface 104, namely, the overlapping area needs to be sensed using theimaging units 112 to grasp how the projection overlaps.

For example, in the case of FIG. 4, a projection area on the projectionsurface 104 responsible by the projection unit 111-1 of the projectionimaging apparatus 102-1 (a range where the projection unit 111-1projects an image) spans in a range of P0L to P0R. Likewise, aprojection area on the projection surface 104 responsible by theprojection unit 111-2 of the projection imaging apparatus 102-2 (a rangewhere the projection unit 111-2 projects an image) spans in a range ofP1L to P1R. That is, a range indicated by a double-headed arrow 130 (arange of P1L to P0R) is applicable. This overlapping area is simplyspecified.

This means the same as estimating the relative posture (a rotationcomponent (R) and a translation component (T)) between the projectionunit 111 and the imaging unit 112. Additionally, information necessaryfor estimating the relative posture is the correspondence relationshipof pixels between the projection unit 111 and the imaging unit 112. Inthe case of the example in FIG. 4, it is only required to find acorrespondence relationship between a pixel of the projection unit 111-2and a pixel of the imaging unit 112-1 indicated by a double-headed arrow131 or a correspondence relationship between a pixel of the projectionunit 111-1 and a pixel of the imaging unit 112-2 indicated by adouble-headed arrow 132, or alternatively, to find both thereof.

Furthermore, as illustrated in FIG. 5, the correspondence relationshipbetween pixels can be expressed as which pixel of the imaging unit 112is a pixel to which a certain pixel of the projection unit 111corresponds. That is, for example, it is assumed in FIG. 5 that lightemitted from the projection unit 111-1 (arrow 133) is reflected at X onthe projection surface 104 and received by the imaging unit 112-2 (arrow134). If it is possible to grasp a correspondence relationship between apixel of the projection unit 111 that have emitted light and a pixel ofthe imaging unit 112 that have received this light as described above(namely, which pixel of the projection unit 111 is a pixel to which eachpixel of the imaging unit 112 corresponds), the overlapping area can bedetected.

As a method for acquiring such a correspondence relationship of pixels,for example, there has been a method using a gray code. In the case ofthis method, for example, predetermined pattern images as illustrated inA of FIG. 6 are projected from the projection unit 111 while beingswitched in time series such that each pattern is imaged by the imagingunit 112. Then, when the imaging of all the patterns is completed, “1”(white) or “0” (black) in each Imaged pattern is detected for each pixelof the imaging unit 112 and, as illustrated in B of FIG. 6, the positionof a projector pixel is acquired by decoding such patterns of “1” and“0”. With these processes, the correspondence relationship of the pixelscan be acquired.

<Online Sensing>

However, in this method, it has been necessary to project the patternimages as illustrated in FIG. 6 as examples in order to acquire thecorrespondence relationship between the pixels and the image projectionmust be interrupted each time. Namely, viewers watching the projectionimage had to be prevented from viewing. Therefore, a technology called“online sensing” for acquiring the correspondence relationship betweenpixels while an image such as contents is being projected isconceivable. As this online sensing, for example, a technique ofembedding a gray code or the like in the projection image, a techniqueutilizing an image feature amount such as SIFT, and a techniqueutilizing invisible light such as Infrared (infrared light) areconceivable.

However, in any case of these methods, the correction information hasbeen updated for all the projection imaging apparatuses 102. That is, asdescribed above, all items of processing such as the calculation of apixel correspondence relationship, the posture estimation for theprojection imaging apparatus 102 (projection unit 111), and the shapeestimation for the projection surface 104 have been performed for allthe projection imaging apparatuses 102. For example, even in a casewhere the postures of some projection imaging apparatuses 102 havechanged, all of these items of the processing have been performed forall the projection imaging apparatuses 102. Therefore, there has been arisk that unnecessary processing is performed, for example, theprocessing becomes redundant for the projection imaging apparatus 102whose position has not changed (in which position misalignment does notoccur). That is, there has been a risk that the processing amount ofprocessing related to the update of the correction information used forgeometric correction and the like unnecessarily increases and theprocessing time unnecessarily increases. In addition, there has been arisk that, as the number of the projection imaging apparatuses 102 to becaused to collaborate increases, the processing amount (processing time)therefor unnecessarily increases.

As described above, such processing related to the update of thecorrection information used for geometric correction and the like can beperformed using the online sensing while an image such as contents isbeing projected but, in some cases, there even has been a possibilitythat the image quality of the projection image is reduced due to theinfluence of this processing on the projection of the image such ascontents. For example, in a case where a pattern image is projected,since this pattern image is projected so as to be superimposed on animage such as contents, there has been a possibility that a picture,switching, and so on of the pattern image are visible to a viewerdepending on the contents of the image such as contents. There has beena possibility that such influence increases as the processing time ofthe processing related to the update of the correction information asdescribed above increases.

<Selection of Processing according to Change Factor>

Therefore, as a table illustrated in FIG. 7, processing to be executedas the processing related to the update of the correction informationused for geometric correction and the like is selected according to achange factor.

For example, in the case of processing (initial sensing) related to theupdate of the correction information used for geometric correction andthe like performed prior to projecting an image such as contents,neither a corresponding point of pixels between the projection unit 111and the imaging unit 112 for a relation between the projection imagingapparatuses 102, the posture of the projection imaging apparatus 102,nor the shape of the projection surface 104 is clear for all theprojection imaging apparatuses 102. Therefore, the control apparatus 101performs each processing of detecting the corresponding point of pixelsbetween the projection unit 111 and the imaging unit 112 for a relationbetween the projection imaging apparatuses 102, estimating the postureof the projection imaging apparatus 102, and estimating the shape of theprojection surface 104 for all the projection imaging apparatuses 102.That is, in the initial sensing, the control apparatus 101 executes allitems of the processing related to the update of the correctioninformation used for geometric correction and the like for all theprojection imaging apparatuses 102.

Additionally, for example, in a case where the postures of all theprojection imaging apparatuses 102 change during the projection of animage such as contents, the corresponding point of pixels between theprojection unit 111 and the imaging unit 112 for a relation between theprojection imaging apparatuses 102, the posture of the projectionimaging apparatus 102, and the shape of the projection surface 104 areall unclear for all the projection imaging apparatuses 102. Therefore,the control apparatus 101 performs each processing of detecting thecorresponding point of pixels between the projection unit 111 and theimaging unit 112 for a relation between the projection imagingapparatuses 102, estimating the posture of the projection imagingapparatus 102, and estimating the shape of the projection surface 104for all the projection imaging apparatuses 102. That is, in a case wherethe postures of all the projection imaging apparatuses 102 change duringthe projection of an image such as contents, the control apparatus 101executes all items of the processing related to the update of thecorrection information used for geometric correction and the like forall the projection imaging apparatuses 102 similarly to the case of theinitial sensing.

In contrast to this, for example, in a case where the posture of a partof the projection imaging apparatuses 102 changes during the projectionof an image such as contents, the corresponding point of pixels betweenthe projection unit 111 and the imaging unit 112 for a relation betweenthe projection imaging apparatuses 102, the posture of the projectionimaging apparatus 102, and the shape of the projection surface 104 allcontinue to be clear for the remaining part of the projection imagingapparatuses 102. That is, the control apparatus 101 can execute onlyprocessing related to the part of the projection imaging apparatuses 102whose posture has changed. Therefore, in a case where the postures ofall the projection imaging apparatuses 102 change during the projectionof an image such as contents, the control apparatus 101 detects thecorresponding point of pixels between the projection unit 111 and theimaging unit 112 for a relation between the projection imagingapparatuses 102 only for a part related to the projection imagingapparatuses 102 whose postures have changed. The control apparatus 101also estimates the posture only for the projection imaging apparatuses102 whose postures have changed. In addition, the control apparatus 101estimates the shape of the projection surface 104 only for a part of theprojection surface 104 in which the projection by the projection imagingapparatuses 102 whose postures have changed is performed and whose shapeis unknown.

For example, as in A of FIG. 8, it is assumed that the projectionimaging apparatus 102-1 and the projection imaging apparatus 102-2 arearranged in such a manner that the projection area on the projectionsurface 104 responsible by the projection unit 111-1 and the projectionarea on the projection surface 104 responsible by the projection unit111-2 overlap at a portion indicated by a double-headed arrow 141.

For example, as in B of FIG. 8, when the projection imaging apparatus102-1 moves, the projection area thereof changes. In this case, aportion indicated by a double-headed arrow 142 serves as the overlappingarea and the overlapping area is narrower than the case of A in FIG. 8.

In such a case, the control apparatus 101 simply performs onlyprocessing related to the projection imaging apparatus 102-1 that hasmoved. For example, as illustrated in C of FIG. 8, if the pixelcorresponding point between the projection unit 111-2 of the projectionimaging apparatus 102-2 and the imaging unit 112-1 of the projectionimaging apparatus 102-1 is detected, the control apparatus 101 canestimate the relative posture between the projection unit 111-2 and theimaging unit 112-1 as indicated by a double-headed arrow 145 illustratedin D of FIG. 8. The projection imaging apparatus 102 has already beencalibrated and the relative posture between the projection unit 111-1and the imaging unit 112-1 is known. Therefore, also the relativeposture between the projection unit 111-2 and the projection unit 111-1is estimated as indicated by a double-headed arrow 146 illustrated in Dof FIG. 8. That is, by detecting the pixel corresponding point betweenthe projection unit 111-2 and the imaging unit 112-1, the relativeposture between the projection imaging apparatuses 102 is estimated.

Meanwhile, for example, in a case where the projection imaging apparatus102-1 and the projection imaging apparatus 102-2 are arranged as in A ofFIG. 9, assuming that the initial sensing has already been carried out,it is understood, from the projection area on the projection surface 104responsible by the projection unit 111-1 and the projection area on theprojection surface 104 responsible by the projection unit 111-2, that arange on the projection surface 104 indicated by a double-headed arrow151 is at least known.

For example, when the projection imaging apparatus 102-1 moves as in Bof FIG. 9, the projection area thereof changes from the state in A ofFIG. 9 but is still located within a range indicated by thedouble-headed arrow 151. Therefore, since the shape of the projectionarea on the projection surface 104 responsible by the projection imagingapparatus 102-1 after the movement is known, the shape estimation forthe projection surface 104 is unnecessary.

In contrast to this, when the projection imaging apparatus 102-1 movesas in C of FIG. 9, the projection area thereof extends outside a rangeindicated by the double-headed arrow 151. That is, there is apossibility that a range of the projection surface 104 indicated by adouble-headed arrow 152 in C of FIG. 9 is unknown. When the shape ofthis range is assumed to be unknown, the control apparatus 101 simplyestimates the shape of the projection surface 104 only for this partwhose shape is unknown.

As described thus far, by performing the processing related to theupdate of the correction information used for geometric correction andthe like (the detection of the corresponding point of pixels between theprojection unit 111 and the imaging unit 112 for a relation between theprojection imaging apparatuses 102, the posture estimation for theprojection imaging apparatus 102, and the shape estimation for theprojection surface 104), the control apparatus 101 can suppress theexecution of unnecessary processing. That is, the control apparatus 101can suppress an increase in processing amount (processing time) of theprocessing related to the update of the correction information used forgeometric correction and the like.

Returning to FIG. 7, for example, in a case where the shape of theprojection surface 104 changes during the projection of an image such ascontents, the postures of all the projection imaging apparatuses 102 areknown. Meanwhile, as for the corresponding point of pixels between theprojection unit 111 and the imaging unit 112 for a relation between theprojection imaging apparatuses 102, a corresponding point according to apart of the projection surface 104 whose shape has changed becomesunclear. In addition, the shape of the projection surface 104 is unclearfor a part thereof whose shape has changed. Therefore, the controlapparatus 101 detects the corresponding point of pixels between theprojection unit 111 and the imaging unit 112 for a relation between theprojection imaging apparatuses 102 only for a part related to theprojection imaging apparatus 102 whose corresponding point has changed.Furthermore, the control apparatus 101 estimates the shape of theprojection surface 104 only for a part of the projection surface 104whose shape becomes unknown due to the shape change. Note that, in thiscase, the posture estimation for the projection imaging apparatus 102 isunnecessary and thus is omitted.

As described above, by performing the processing related to the updateof the correction information used for geometric correction and the like(the detection of the corresponding point of pixels between theprojection unit 111 and the imaging unit 112 for a relation between theprojection imaging apparatuses 102, the posture estimation for theprojection imaging apparatus 102, and the shape estimation for theprojection surface 104), the control apparatus 101 can suppress theexecution of unnecessary processing. That is, the control apparatus 101can suppress an increase in processing amount (processing time) of theprocessing related to the update of the correction information used forgeometric correction and the like.

Meanwhile, for example, in a case where the posture of a part of theprojection imaging apparatuses 102 changes and the shape of theprojection surface 104 also changes during the projection of an imagesuch as contents, corresponding points of pixels related to the part ofthe projection imaging apparatuses 102 whose posture has changed change.In addition, a corresponding point of pixels corresponding to a part ofthe projection surface 104 whose shape has changed also changes. Theother corresponding points do not change. Therefore, in this case, thecontrol apparatus 101 detects the corresponding point of pixels betweenthe projection unit 111 and the imaging unit 112 for a relation betweenthe projection imaging apparatuses 102 only for a part related to theprojection imaging apparatus 102 whose posture has changed and a partrelated to the projection imaging apparatus 102 whose correspondingpoint has changed.

Furthermore, in this case, the posture changes only for the part of theprojection imaging apparatuses 102 whose posture has changed but thepostures of the other projection imaging apparatuses 102 do not change.Therefore, in this case, the control apparatus 101 estimates the postureonly for the projection imaging apparatus 102 whose posture has changed.

Furthermore, the control apparatus 101 estimates the shape of theprojection surface 104 only for a part of the projection surface 104whose shape is unknown. It is not necessary to estimate the shape of apart whose shape is known even in a case where any of the posture changein the projection imaging apparatus 102 and the shape change in theprojection surface 104 acts as a factor. That is, the shape estimationis required to be performed only for an unknown part.

As described thus far, by performing the processing related to theupdate of the correction information used for geometric correction andthe like (the detection of the corresponding point of pixels between theprojection unit 111 and the imaging unit 112 for a relation between theprojection imaging apparatuses 102, the posture estimation for theprojection imaging apparatus 102, and the shape estimation for theprojection surface 104), the control apparatus 101 can suppress theexecution of unnecessary processing. That is, the control apparatus 101can suppress an increase in processing amount (processing time) of theprocessing related to the update of the correction information used forgeometric correction and the like.

That is, as described thus far, by selecting processing to be executeddepending on the change factor, only necessary processing according toeach processing can be performed, whereby the control apparatus 101 cansuppress an increase in processing amount (processing time) of theprocessing related to the update of the correction information used forgeometric correction and the like.

<Configuration of Control Apparatus>

FIG. 10 is a block diagram illustrating a main configuration example ofthe control apparatus 101 which is an embodiment of the informationprocessing apparatus to which the present technology is applied.

As illustrated in FIG. 10, the control apparatus 101 has a control unit201. Processing related to the control of each member of the controlapparatus 101 and the projection imaging apparatus 102, and so on areperformed therein.

An input/output interface 210 is connected to the control unit 201 via abus. An input unit 211, an output unit 212, a storage unit 213, acommunication unit 214, and a drive 215 are connected to theinput/output interface 210.

The input unit 211 is constituted by an input device that acceptsexternal information such as user input. For example, the input unit 211includes an operation button, a touch panel, a camera, a microphone, andan input terminal. In addition, various sensors such as an accelerationsensor, a light sensor, and a temperature sensor may be included in theinput unit 211.

The output unit 212 is constituted by an output device that outputsinformation such as an image and a sound. For example, the output unit212 includes a display, a speaker, and an output terminal.

For example, the storage unit 213 is constituted by a hard disk, a RAMdisk, or a non-volatile memory. For example, the communication unit 214is constituted by a network interface. For example, the communicationunit 214 is connected to the communication cable 103 and communicateswith another apparatus connected via the communication cable 103. Thedrive 215 drives a removable medium 221 such as a magnetic disk, anoptical disc, a magneto-optical disk, or a semiconductor memory.

The control unit 201 has a processing control unit 231, a projectioncontrol unit 232, an imaging control unit 233, a sensor control unit234, an image projection unit 235, a corresponding point changedetection unit 236, a sensor information change detection unit 237, achange determination unit 238, a corresponding point detection unit 239,a posture estimation unit 240, a projection surface shape estimationunit 241, and a correction information update unit 242.

The processing control unit 231 performs processing related to thecontrol of processing related to correction information update. Theprojection control unit 232 performs processing related to the controlof the projection unit 111. The imaging control unit 233 performsprocessing related to the control of the imaging unit 112. The sensorcontrol unit 234 performs processing related to the control of a sensor(sensor unit 312) included in each projection imaging apparatus 102.

The image projection unit 235 performs processing related to theprojection of an image such as contents. The corresponding point changedetection unit 236 performs processing related to the detection of achange in the corresponding point of pixels between the projection unit111 and the imaging unit 112 for a relation between the projectionimaging apparatuses 102. The sensor information change detection unit237 performs processing related to the detection of a change in sensorinformation. The change determination unit 238 performs processingrelated to the determination as to what kind of information has changed.

The corresponding point detection unit 239 performs processing relatedto the detection of the corresponding point of pixels between theprojection unit 111 and the imaging unit 112 for a relation between theprojection imaging apparatuses 102. The posture estimation unit 240performs processing related to the posture estimation for the projectionimaging apparatus 102. The projection surface shape estimation unit 241performs processing related to the shape estimation for the projectionsurface 104. The correction information update unit 242 performsprocessing related to the update of the correction information.

Note that each of these processing units can be also realized bysoftware. In that case, the control unit 201 has a configuration capableof executing a program and processing data, such as a central processingunit (CPU), a read only memory (ROM), and a random access memory (RAM).Then, the control unit 201 performs various types of processing by, forexample, loading a program read from the storage unit 213 or the likeinto a built-in RAM or the like to execute and processing data read fromthe storage unit 213 or the like. In this case, the respectiveprocessing units of the processing control unit 231 to the correctioninformation update unit 242 are realized as functional blocks. That is,the control unit 201 executes a program to realize various functions ofthe processing control unit 231 to the correction information updateunit 242.

<Appearance of Projection Imaging Apparatus>

FIG. 11 illustrates an example of the appearance of the projectionimaging apparatus 102. As described above, the projection imagingapparatus 102 has the projection unit 111 and the imaging unit 112 andoptical devices such as a projection port (lens mechanism) 301 forprojecting an image and an imaging port (lens mechanism) 302 for imaginga subject are provided in a cabinet thereof. In addition, the projectionimaging apparatus 102 may be of any dimension but, for example, may be aportable (compact) apparatus. In that case, a battery 303 may beprovided in the cabinet of the projection imaging apparatus 102 asillustrated in FIG. 11. By providing the battery 303, the projectionimaging apparatus 102 can be driven without an external power supply andthus, the degree of freedom of the installation position thereof can beenhanced.

<Projection Imaging Apparatus>

FIG. 12 is a block diagram illustrating a main configuration example ofthe projection imaging apparatus 102.

As illustrated in FIG. 12, the projection imaging apparatus 102 has acontrol unit 311, the projection unit 111, the imaging unit 112, thesensor unit 312, an input unit 321, an output unit 322, a storage unit323, a communication unit 324, and a drive 325.

The control unit 311 has, for example, a CPU, a ROM, and a RAM tocontrol each processing unit within the apparatus and execute varioustypes of processing such as image processing required for the controlthereof.

The sensor unit 312 has a sensor capable of detecting a change inposture of the projection imaging apparatus 102, such as an accelerationsensor or each velocity sensor. This sensor may sense any kind ofinformation as long as the sensor can detect a change in posture of theprojection imaging apparatus 102. The sensor unit 312 senses apredetermined parameter under the control of the control unit 311 andsupplies a result of the detection to the control unit 311.

The projection unit 111 performs processing related to the projection ofan image under the control of the control unit 311. For example, theprojection unit 111 projects an image supplied from the control unit 311on the outside of the projection imaging apparatus 102 (for example, theprojection surface 104). That is, the projection unit 111 realizes aprojection function. The projection unit 111 uses a laser beam as alight source and projects an image by scanning this laser beam using aMEMS mirror. As a matter of course, the light source of the projectionunit 111 is arbitrary and not limited to the laser beam. The lightsource may be a light emitting diode (LED), xenon, or the like. Detailsof the projection unit 111 will be described later.

Under the control of the control unit 311, the imaging unit 112 images asubject outside the apparatus to generate a captured image and suppliesthis captured image to the control unit 311. That is, the imaging unit112 realizes an imaging function. For example, the imaging unit 112captures a projection image projected on the projection surface 104 bythe projection unit 111.

The input unit 321 is constituted by an input device that acceptsexternal information such as user input. For example, the input unit 321includes an operation button, a touch panel, a camera, a microphone, andan input terminal. In addition, various sensors such as a light sensorand a temperature sensor may be included in the input unit 321.

The output unit 322 is constituted by an output device that outputsinformation such as an image and a sound. For example, the output unit322 includes a display, a speaker, and an output terminal.

For example, the storage unit 323 is constituted by a hard disk, a RAMdisk, or a non-volatile memory. For example, the communication unit 324is constituted by a network interface. For example, the communicationunit 324 is connected to the communication cable 103 and communicateswith another apparatus connected via the communication cable 103. Thedrive 325 drives a removable medium 331 such as a magnetic disk, anoptical disc, a magneto-optical disk, or a semiconductor memory.

<Projection Unit>

FIG. 13 is a block diagram illustrating a main configuration example ofthe projection unit ill. As illustrated in FIG. 13, the projection unit111 has a video processor 351, a laser driver 352, a laser output unit353-1, a laser output unit 353-2, a laser output unit 353-3, a mirror354-1, a mirror 354-2, a mirror 354-3, a micro electro mechanicalsystems (MEMS) driver 355, and a MEMS mirror 356.

The video processor 351 holds an image supplied from the control unit311 and performs necessary image processing on this image. The videoprocessor 351 supplies the image to be projected to the laser driver 352and the MEMS driver 355.

The laser driver 352 controls the laser output units 353-1 to 353-3 soas to project the image supplied from the video processor 351. The laseroutput units 353-1 to 353-3 output laser beams, for example, havingdifferent colors (wavelength ranges) from each other such as red, blue,and green. That is, the laser driver 352 controls the laser output ofeach color so as to project the image supplied from the video processor351. Note that, in a case where it is not necessary to distinguish thelaser output units 353-1 to 353-3 from each other when described, theselaser output units are referred to as laser output units 353.

The mirror 354-1 reflects the laser beam output from the laser outputunit 353-1 to guide to the MEMS mirror 356. The mirror 354-2 reflectsthe laser beam output from the laser output unit 353-2 to guide to theMEMS mirror 356. The mirror 354-3 reflects the laser beam output fromthe laser output unit 353-3 to guide to the MEMS mirror 356. Note that,in a case where it is not necessary to distinguish the mirrors 354-1 to354-3 from each other when described, these mirrors are referred to asmirrors 354.

The MEMS driver 355 controls the driving of a mirror of the MEMS mirror356 so as to project the image supplied from the video processor 351.The MEMS mirror 356 drives the mirror (reflector) attached on top of theMEMS in accordance with the control of the MEMS driver 355 to scan thelaser beam of each color as illustrated in FIG. 14 as an example. Thislaser beam is output from the projection port to the outside of theapparatus and, for example, the projection surface 104 is irradiatedtherewith. As a result, the image supplied from the video processor 351is projected on the projection surface 104.

Note that the example in FIG. 13 has described three laser output units353 provided to output the laser beams of three colors but the number oflaser beams (or the number of colors) is arbitrary. For example, thenumber of laser output units 353 may be four or more or may be two orless. That is, the number of laser beams output from the projectionimaging apparatus 102 (projection unit 111) may be two or less or may befour or more. Additionally, the number of colors of laser beams outputfrom the projection imaging apparatus 102 (projection unit 111) is alsoarbitrary and may be two or less or may be four or more. Theconfigurations of the mirror 354 and the MEMS mirror 356 are alsoarbitrary and are not limited to the example in FIG. 13. As a matter ofcourse, a scanning pattern of the laser beam is arbitrary.

<Flow of Correction Information Update Processing>

Next, an example of a flow of correction information update processingexecuted by the control unit 201 of the control apparatus 101 will bedescribed with reference to a flowchart in FIG. 15.

The control unit 201 of the control apparatus 101 first performs theinitial sensing. As described above with reference to FIG. 7, in theinitial sensing, all items of the processing related to the update ofthe correction information used for geometric correction and the likeare executed for all the projection imaging apparatuses 102. After theinitial sensing is performed, the image projection unit 235 controls theprojection imaging apparatus 102 via the projection control unit 232 toproject an image such as contents. In accordance with this control, thecontrol unit 311 of the projection imaging apparatus 102 controls theprojection unit 111 to project an image such as contents supplied fromthe control apparatus 101 on the projection surface 104.

In parallel with this image projection, the control apparatus 101performs the correction information update processing to update thecorrection information used for geometric correction and the like asnecessary. This correction information update processing is executed foreach projection imaging apparatus 102. Note that the correctioninformation update processing may be perform on the projection imagingapparatus 102 one by one or the correction information update processingmay be performed on the plurality of projection imaging apparatuses 102in parallel. In addition, a processing order thereof is arbitrary andmay be a fixed order or random. The correction information updateprocessing is only required to be performed on all the projectionimaging apparatuses 102.

Once such correction information update processing is started, thecorresponding point change detection unit 236 detects, in step S101, achange in the corresponding point of pixels between the projection unit111 and the imaging unit 112 for a relation between a current projectionimaging apparatus 102 being processed and another projection imagingapparatus 102. The corresponding point change detection unit 236controls the projection control unit 232 and the imaging control unit233 to perform this detection processing. That is, the correspondingpoint change detection unit 236 acquires an image projected by theprojection unit 111 from the projection control unit 232. In addition,the corresponding point change detection unit 236 causes the projectionimaging apparatus 102 via the imaging control unit 233 to image theprojection surface 104 and acquires a captured image thereof. Thecorresponding point change detection unit 236 aligns the positions ofthese images to monitor a change in the corresponding point of pixelsbetween the projection unit 111 and the imaging unit 112 for a relationbetween the current projection imaging apparatus 102 and the anotherprojection imaging apparatus 102 and, in a case Where the correspondingpoint has changed, detects this change.

In step S102, the sensor information change detection unit 237 detects achange in sensor information obtained by the sensor unit 312 of thecurrent projection imaging apparatus 102. The sensor information changedetection unit 237 controls the sensor control unit 234 to perform thisdetection processing. That is, the sensor information change detectionunit 237 acquires a sensor output from the sensor unit 312 of thecurrent projection imaging apparatus 102 via the sensor control unit234. Then, on the basis of this sensor output, the sensor informationchange detection unit 237 detects a change in sensor information in atime direction.

In step S103, the change determination unit 238 determines whether theposture of the current projection imaging apparatus 102 has changed. Ina case where a change in sensor information is detected in step S102 andit is determined that the posture of the current projection imagingapparatus 102 has changed, the processing proceeds to step S104.

In step S104, the corresponding point detection unit 239 detects thecorresponding point of pixels related to the current projection imagingapparatus 102.

In step S105, the posture estimation unit 240 estimates the posture ofthe current projection imaging apparatus 102 using a processing resultof step S104 (pixel corresponding point detection result). Namely, theposture estimation unit 240 estimates the relative posture of thecurrent projection imaging apparatus 102 with respect to anotherprojection imaging apparatus 102 whose posture has not changed.

In step S106, the projection surface shape estimation unit 241 estimatesthe shape of the projection surface 104. Once the processing in stepS106 is terminated, the processing proceeds to step S110.

Meanwhile, in a case where no change in sensor information is detectedin step S102 and it is determined in step S103 that the posture of thecurrent projection imaging apparatus 102 has not changed, the processingproceeds to step S107.

In step S107, the change determination unit 238 determines whether theshape of the projection surface 104 has changed on the basis of aprocessing result of step S101. For example, in a case where thecorresponding point of pixels related to the current projection imagingapparatus 102 has changed even though the posture of the currentprojection imaging apparatus 102 has not changed, the shape of theprojection surface 104 has changed. In a case where it is determinedthat the shape of the projection surface 104 has changed in this manner,the processing proceeds to step S108.

In step S108, the corresponding point detection unit 239 detects thecorresponding point of pixels related to the current projection imagingapparatus 102. Note that, in this case, since the posture of the currentprojection imaging apparatus 102 has not changed, the estimation of theposture is omitted.

In step S109, the projection surface shape estimation unit 241 estimatesthe shape of the projection surface 104. Once the processing in stepS109 is terminated, the processing proceeds to step S110.

In step S110, in a case where information has been updated, added, ordeleted during the respective items of the above processing, thecorrection information update unit 242 updates the correctioninformation so as to reflect the update, addition, or deletion of theinformation in the correction information.

Once the processing in step S110 is terminated, the correctioninformation update processing is terminated. In addition, in a casewhere it is determined in step S107 that the shape of the projectionsurface 104 has not changed, the correction information updateprocessing is terminated. In this case, the posture of the currentprojection imaging apparatus 102 has not changed and also the shape of apart of the projection surface 104 associated with the currentprojection imaging apparatus 102 has not changed. Therefore, theprocessing related to the update of the correction information used forgeometric correction and the like is all omitted.

As described above, in step S104, the corresponding point detection unit239 detects the corresponding point between a pixel of the projectionunit 111 and a pixel of the imaging unit 112 for a relation between theprojection imaging apparatus 102 whose posture has changed and theprojection imaging apparatus 102 whose posture has not changed. Then, instep S105, the posture estimation unit 240 estimates the relativeposture of the projection imaging apparatus 102 whose posture haschanged with respect to the projection imaging apparatus 102 whoseposture has not changed on the basis of that detected correspondingpoint of pixels.

Therefore, the control apparatus 101 can suppress the execution ofunnecessary processing by, for example, omitting the estimation of therelative posture between the projection imaging apparatuses 102 whosepostures have not changed. That is, the control apparatus 101 cansuppress an increase in processing amount (processing time) of theprocessing related to the update of the correction information used forgeometric correction and the like.

In addition, in step S101, the corresponding point change detection unit236 detects a change in the corresponding point between a pixel of theprojection unit 111 and a pixel of the imaging unit 112 of theprojection imaging apparatuses 102. Then, in step S102, the sensorinformation change detection unit 237 detects a posture change in theprojection imaging apparatus on the basis of the sensor output from thesensor unit which is included in the projection imaging apparatus 102and detects at least one of the position and the direction. Furthermore,in step S103, on the basis of the detection result of the correspondingpoint change detection unit 236 and the detection result of the sensorinformation change detection unit 237, the change determination unit 238determines whether the posture of the projection imaging apparatus 102has changed.

Then, in step S104, the corresponding point detection unit 239 detectsthe corresponding point for a relation between the projection imagingapparatus 102 whose posture is determined by the change determinationunit 238 to have changed and the projection imaging apparatus 102 whoseposture is determined by the change determination unit 238 to have notchanged. In addition, in step S105, the posture estimation unit 240estimates the relative posture of the projection imaging apparatus 102whose posture is determined by the change determination unit 238 to havechanged, with respect to the projection imaging apparatus 102 whoseposture is determined by the change determination unit 238 to have notchanged.

Therefore, the control apparatus 101 can suppress the execution ofunnecessary processing by, for example, omitting the estimation of therelative posture between the projection imaging apparatuses 102 whosepostures have not changed. That is, the control apparatus 101 cansuppress an increase in processing amount (processing time) of theprocessing related to the update of the correction information used forgeometric correction and the like.

Additionally, in step S104, the corresponding point detection unit 239further detects the corresponding point between a pixel of theprojection unit 111 and a pixel of the imaging unit 112 of theprojection imaging apparatus 102 whose posture has changed. Then, instep S106, the projection surface shape estimation unit 241 estimatesthe shape of the projection surface 104 on the basis of thecorresponding point detected by the corresponding point detection unit239 for a relation between a pixel of the projection unit 111 and apixel of the imaging unit 112 of the projection imaging apparatus 102whose posture has changed.

Therefore, the control apparatus 101 can estimate the shape byspecifying an area of the projection image having a possibility ofincluding a part whose shape is unknown, such as a projection rangeresponsible by the projection imaging apparatus 102 whose posture haschanged, thereby being able to suppress the execution of unnecessaryprocessing by, for example, omitting the shape estimation for a partwhose shape is obviously known. That is, the control apparatus 101 cansuppress an increase in processing amount (processing time) of theprocessing related to the update of the correction information used forgeometric correction and the like.

In addition, in step S108, the corresponding point detection unit 239further detects the corresponding point between a pixel of theprojection unit 111 and a pixel of the imaging unit 112 of theprojection imaging apparatus 102 whose posture has not changed.Furthermore, the projection surface shape estimation unit 241 is causedto estimate the shape of the projection surface 104 on the basis of thecorresponding point detected by the corresponding point detection unit239 for a relation between a pixel of the projection unit 111 and apixel of the imaging unit 112 of the projection imaging apparatus 102whose posture has not changed.

By processing in such a manner, the control apparatus 101 detects ashape change in the projection surface 104 from a change in thecorresponding point of pixels even in a case where the posture of theprojection imaging apparatus 102 does not change and thus can estimatethe shape of the projection surface that has changed. Therefore, theexecution of unnecessary processing can be suppressed by, for example,omitting the shape estimation for a part whose shape is obviously known.That is, the control apparatus 101 can suppress an increase inprocessing amount (processing time) of the processing related to theupdate of the correction information used for geometric correction andthe like.

In other words, the corresponding point detection unit 239 of thecontrol apparatus 101 detects the corresponding point between a pixel ofthe projection unit 111 and a pixel of the imaging unit 112 of theprojection imaging apparatus 102 whose posture has not changed and theprojection surface shape estimation unit 241 estimates the shape of theprojection surface 104 on the basis of the corresponding point detectedby this corresponding point detection unit 239 for a relation between apixel of the projection unit 111 and a pixel of the imaging unit 112 ofthe projection imaging apparatus 102 whose posture has not changed.

By processing in this manner, the control apparatus 101 can specify apart of the projection surface 104 whose shape has changed and estimatethe shape of the projection surface 104 for this part. Therefore, theexecution of unnecessary processing can be suppressed by, for example,omitting the shape estimation for a part whose shape is obviously known.That is, the control apparatus 101 can suppress an increase inprocessing amount (processing time) of the processing related to theupdate of the correction information used for geometric correction andthe like.

<Flow of Corresponding Point Detection Processing>

Next, an example of a flow of corresponding point detection processingexecuted in step S104 of FIG. 15 will be described with reference to aflowchart in FIG. 16.

Once the corresponding point detection processing is started, thecorresponding point detection unit 239 controls, in step S131, theprojection unit 111 of another projection imaging apparatus 102 which isnot the processing target via the projection control unit 232 to projecta predetermined pattern image.

In step S132, the corresponding point detection unit 239 controls theimaging unit 112 of the current projection imaging apparatus 102 via theimaging control unit 233 to capture the projection image (pattern image)projected by the processing in step S131.

In step S133, the corresponding point detection unit 239 detects thecorresponding point between a pixel of the projection unit 111 of theanother projection imaging apparatus 102 and a pixel of the imaging unit112 of the current projection imaging apparatus 102. That is, on thebasis of the pattern image projected during the processing in step S131and the captured image (captured image including the pattern image)obtained through the imaging during the processing in step S132, thecorresponding point detection unit 239 detects this corresponding point.

In step S134, the corresponding point detection unit 239 controls theprojection unit 111 of the current projection imaging apparatus 102 viathe projection control unit 232 to project a predetermined patternimage.

In step S135, the corresponding point detection unit 239 controls theimaging unit 112 of the current projection imaging apparatus 102 via theimaging control unit 233 to capture the projection image (pattern image)projected by the processing in step S134.

In step S136, the corresponding point detection unit 239 detects thecorresponding point between a pixel of the projection unit 111 and apixel of the imaging unit 112 of the current projection imagingapparatus 102. That is, on the basis of the pattern image projectedduring the processing in step S134 and the captured image (capturedimage including the pattern image) obtained through the imaging duringthe processing in step S135, the corresponding point detection unit 239detects this corresponding point.

When the pixel corresponding point is detected as described above, thecorresponding point detection processing is terminated and theprocessing returns to FIG. 15.

As described thus far, the corresponding point detection unit 239detects the corresponding point of pixels between the imaging unit 112of the projection imaging apparatus 102 whose posture has changed andthe projection unit 111 of the projection imaging apparatus 102 whoseposture has not changed and also detects the corresponding point ofpixels between the projection unit 111 and the imaging unit 112 of theprojection imaging apparatus 102 whose posture has changed. As a result,in the correction information update processing (FIG. 15), the postureestimation unit 240 can estimate the posture of the projection imagingapparatus 102 whose posture has changed and also can estimate the shapeof the projection surface 104. That is, the control apparatus 101 cansuppress an increase in processing amount (processing time) of theprocessing related to the update of the correction information used forgeometric correction and the like.

<Flow of Projection Surface Shape Estimation Processing>

Next, an example of a flow of projection surface shape estimationprocessing executed in step S106 of FIG. 15 will be described withreference to a flowchart in FIG. 17.

Once the projection surface shape estimation processing is started, theprojection surface shape estimation unit 241 controls the projectioncontrol unit 232 in step S151 to estimate a projection range on theprojection surface 104 on which the projection unit 111 of the currentprojection imaging apparatus 102 projects an image. For example, theprojection surface shape estimation unit 241 estimates theabove-mentioned projection range on the basis of the posture of thecurrent projection imaging apparatus 102 estimated by the processing instep S105 (FIG. 15), and so on.

In step S152, the projection surface shape estimation unit 241determines whether an unknown part of the projection surface 104 isincluded in this estimated projection range. In a case where it isdetermined that an unknown part is included, the processing proceeds tostep S153.

In step S153, the projection surface shape estimation unit 241 estimatesthe shape of the projection surface 104 for this unknown part. Forexample, on the basis of the pixel corresponding point between theprojection unit 111 and the imaging unit 112 of the current projectionimaging apparatus 102 detected by the processing in step S136 (FIG. 16),the projection surface shape estimation unit 241 estimates the shape ofthe projection surface 104 for this unknown part.

Once the shape of the projection surface 104 is estimated, theprojection surface shape estimation processing is terminated and theprocessing returns to FIG. 15. In addition, in a case where it isdetermined in step S152 that an unknown part is not included in theprojection range, the projection surface shape estimation processing isterminated and the processing returns to FIG. 15. That is, in this case,the processing in step S153 (projection surface shape estimation) isomitted.

As described thus far, the projection surface shape estimation unit 241estimates the shape of the projection surface 104 for a part of theprojection surface 104 whose shape is unknown within a range on which animage is projected by the projection unit 111 of the projection imagingapparatus 102 whose posture has changed.

Therefore, the control apparatus 101 can further suppress an increase inprocessing amount (processing time) of the processing related to theupdate of the correction information used for geometric correction andthe like.

<Flow of Corresponding Point Detection Processing>

Next, an example of a flow of corresponding point detection processingexecuted in step S108 of FIG. 15 will be described with reference to aflowchart in FIG. 18.

Once the corresponding point detection processing is started, thecorresponding point detection unit 239 controls, in step S171, theprojection unit 111 of the current projection imaging apparatus 102 viathe projection control unit 232 to project a predetermined patternimage.

In step S172, the corresponding point detection unit 239 controls theimaging unit 112 of the current projection imaging apparatus 102 via theimaging control unit 233 to capture the projection image (pattern image)projected by the processing in step S171.

In step S173, the corresponding point detection unit 239 detects thecorresponding point between a pixel of the projection unit 111 and apixel of the imaging unit 112 of the current projection imagingapparatus 102. That is, on the basis of the pattern image projectedduring the processing in step S171 and the captured image (capturedimage including the pattern image) obtained through the imaging duringthe processing in step S172, the corresponding point detection unit 239detects this corresponding point.

When the pixel corresponding point is detected as described above, thecorresponding point detection processing is terminated and theprocessing returns to FIG. 15.

That is, in this case, since the posture of the current projectionimaging apparatus 102 has not changed, the estimation of the posture ofthis projection imaging apparatus 102 is omitted. Therefore, theprocessing related to the detection of the corresponding point of pixelsbetween the projection unit 111 and the imaging unit 112 for a relationbetween the projection imaging apparatuses 102 used for the aboveposture estimation (respective items of the processing in steps S131 toS133 in FIG. 16) is also omitted. Therefore, the control apparatus 101can further suppress an increase in processing amount (processing time)of the processing related to the update of the correction informationused for geometric correction and the like.

<Flow of Projection Surface Shape Estimation Processing>

Next, an example of a flow of projection surface shape estimationprocessing executed in step S109 of FIG. 15 will be described withreference to a flowchart in FIG. 19.

Once the projection surface shape estimation processing is started, theprojection surface shape estimation unit 241 specifies, in step S191, achanged part of the corresponding points of pixels between theprojection unit 111 and the imaging unit 112 of the current projectionimaging apparatus 102 on the basis of a processing result of step S173(FIG. 18).

In step S192, the projection surface shape estimation unit 241 estimatesthe shape of the projection surface 104 for a part of the projectionsurface 104 whose shape is unknown (a changed part of the correspondingpoints of pixels specified in step S191). This processing is executedsimilarly to the case of the processing in step S153 (FIG. 17). Once theshape of the projection surface 104 is estimated, the projection surfaceshape estimation processing is terminated and the processing returns toFIG. 15.

As described thus far, the projection surface shape estimation unit 241estimates the shape of the projection surface 104 for a part thereofwhose shape is unknown. Therefore, the control apparatus 101 can furthersuppress an increase in processing amount (processing time) of theprocessing related to the update of the correction information used forgeometric correction and the like.

By executing each item of the processing as described above, the controlapparatus 101 can perform only processing according to the change factorand thus can suppress an increase in processing amount of the processingrelated to the update of the correction information used for geometriccorrection and the like.

2. Second Embodiment

<Corresponding Point Detection Processing>

Note that the above explanation has described that, when thecorresponding point of pixels between the projection unit 111 and theimaging unit 112 is found for a relation between the projection imagingapparatuses 102, the pattern image is projected from the projection unit111 of another projection imaging apparatus 102 such that the imagingunit 112 of the current projection imaging apparatus 102 captures thisprojection image. However, the projection and the capturing of theimages in this case may be reversed. Namely, the pattern image may beprojected from the projection unit 111 of the current projection imagingapparatus 102 and this projection image may be captured by the imagingunit 112 of another projection imaging apparatus 102. That is, thecorresponding point between a pixel of the imaging unit 112 of anotherprojection imaging apparatus 102 and a pixel of the projection unit 111of the current projection imaging apparatus 102 may be detected. Theposture of the current projection imaging apparatus 102 can be estimatedalso from this corresponding point similarly to the case of the firstembodiment.

An example of a flow of corresponding point detection processingexecuted in step S104 of FIG. 15 in that case will be described withreference to a flowchart in FIG. 20.

Once the corresponding point detection processing is started, acorresponding point detection unit 239 controls, in step S211, aprojection unit 111 of a current projection imaging apparatus 102 via aprojection control unit 232 to project a predetermined pattern image.

In step S212, the corresponding point detection unit 239 controls theimaging unit 112 of the current projection imaging apparatus 102 and theimaging unit 112 of another projection imaging apparatus 102 that is notthe processing target via the imaging control unit 233 to individuallycapture the projection image (pattern image) projected by the processingin step S211.

In step S213, the corresponding point detection unit 239 detects thecorresponding point between a pixel of the imaging unit 112 of theanother projection imaging apparatus 102 and a pixel of the projectionunit 111 of the current projection imaging apparatus 102. That is, onthe basis of the pattern image projected during the processing in stepS211 and the captured images (captured images including the patternimage) obtained through the imaging during the processing in step S212,the corresponding point detection unit 239 detects this correspondingpoint.

In step S214, the corresponding point detection unit 239 detects thecorresponding point between a pixel of the projection unit 111 and apixel of the imaging unit 112 of the current projection imagingapparatus 102. That is, on the basis of the pattern image projectedduring the processing in step S211 and the captured images (capturedimages including the pattern image) obtained through the imaging duringthe processing in step S212, the corresponding point detection unit 239detects this corresponding point.

When the pixel corresponding point is detected as described above, thecorresponding point detection processing is terminated and theprocessing returns to FIG. 15.

That is, the corresponding point detection unit 239 detects thecorresponding point between a pixel of the projection unit 111 of theprojection imaging apparatus 102 whose posture has changed and a pixelof the imaging unit 112 of the projection imaging apparatus 102 whoseposture has not changed. Also in this case, similarly to the case of thefirst embodiment, the control apparatus 101 can perform only processingaccording to the change factor and thus can suppress an increase inprocessing amount of the processing related to the update of thecorrection information used for geometric correction and the like.

3. Third Embodiment

<Configuration Example of Projection Imaging System>

Note that the configuration examples of the projection imaging apparatusand the projection imaging system to which the present technology isapplied is not limited to the examples described above. For example, asin a projection imaging system 400 illustrated in A of FIG. 21, thecontrol apparatus 101 may be omitted. That is, the processing related tothe update of the correction information used for geometric correctionand the like may be performed by an apparatus other than the controlapparatus 101. For example, the projection imaging apparatus 102 mayperform the processing.

In the example in A of FIG. 21, the projection imaging apparatus 102performs the above-described processing related to the update of thecorrection information used for geometric correction and the like. Inthis case, an input image is input to each projection imaging apparatus102 and the correction of the image is performed in each projectionimaging apparatus 102. The projection imaging apparatuses 102 areconnected to each other through a communication cable such as an HDMI(registered trademark) cable and various types of information are sharedamong these projection imaging apparatuses 102. Therefore, anyprojection imaging apparatus 102 can perform the above-describedprocessing related to the update of the correction information used forgeometric correction and the like.

For example, one of the projection imaging apparatuses 102-1 to 102-4may perform the above-described processing related to the update of thecorrection information used for geometric correction and the like so asto share a result of this processing with the other projection imagingapparatuses 102. In addition, the plurality of projection imagingapparatuses 102 may collaborate to perform the above-describedprocessing related to the update of the correction information used forgeometric correction and the like such that a result of this processingis shared by the respective projection imaging apparatuses 102.

As described thus far, even in a case where the projection imagingapparatus performs the processing related to the update of thecorrection information used for geometric correction and the like, theprocessing can be performed in a similar manner to the cases of thefirst embodiment and the second embodiment.

That is, the projection imaging apparatus is configured to include theprojection unit 111 that projects an image on the projection surface104; the imaging unit 112 that images the projection surface 104; thesensor unit 312 that detects at least one of a position and a direction;the change determination unit 238 that determines whether a posture haschanged on the basis of a sensor output from this sensor unit 312; thecorresponding point detection unit 239 that detects, in a case where thechange determination unit 238 determines that the posture has changed, acorresponding point between a pixel of the projection unit 111 and apixel of the imaging unit 112 for a relation with another projectionimaging apparatus 102 which includes the projection unit 111, theimaging unit 112, and the sensor unit 312 and whose posture has notchanged; and the posture estimation unit 240 that estimates a relativeposture with respect to the another projection imaging apparatus 102 onthe basis of the corresponding point detected by the corresponding pointdetection unit 239.

As a result, the control apparatus 101 can perform only processingaccording to the change factor and thus can suppress an increase inprocessing amount of the processing related to the update of thecorrection information used for geometric correction and the like.

Note that the corresponding point detection unit 239 may detect thecorresponding point between a pixel of the projection unit 111 ofanother projection imaging apparatus 102 and a pixel of the own imagingunit 112 of the projection imaging apparatus 102.

Alternatively, the corresponding point detection unit 239 may detect thecorresponding point between a pixel of the own projection unit 111 ofthe projection imaging apparatus 102 and a pixel of the imaging unit 112of another projection imaging apparatus 102.

Additionally, the corresponding point change detection unit 236 thatdetects a change in the corresponding point between a pixel of the ownprojection unit 111 and a pixel of the own imaging unit 112 of theprojection imaging apparatus 102; and the sensor information changedetection unit 237 that detects an own posture change in the projectionimaging apparatus 102 on the basis of a sensor output from the sensorunit 312 may be further included, in which the change determination unit238 may be configured to determine whether an own posture of theprojection imaging apparatus 102 has changed on the basis of a detectionresult of the corresponding point change detection unit 236 and adetection result of the sensor information change detection unit 237.

In addition, the corresponding point detection unit 239 may furtherdetect the corresponding point between a pixel of the own projectionunit 111 and a pixel of the own imaging unit 112 of the projectionimaging apparatus 102. Furthermore, the projection surface shapeestimation unit 241 that estimate the shape of the projection surface104 on the basis of the corresponding point detected by thecorresponding point detection unit 239 for a relation between a pixel ofthe own projection unit 111 and a pixel of the own imaging unit 112 ofthe projection imaging apparatus 102 may be further included.

The projection surface shape estimation unit 241 may estimate the shapeof the projection surface 104 for a part of the projection surface 104whose shape is unknown within a range on which an image is projected bythe own projection unit 111 of the projection imaging apparatus 102.

Meanwhile, the projection unit that projects an image on the projectionsurface; the imaging unit that images the projection surface; the sensorunit 312 that detects at least one of a position and a direction, thechange determination unit 238 that determines whether a posture haschanged on the basis of a sensor output from the sensor unit 312; thecorresponding point detection unit 239 that detects, in a case where thechange determination unit 238 determines that the posture has notchanged, a corresponding point between a pixel of the own projectionunit 111 and a pixel of the own imaging unit 112 of the projectionimaging apparatus 102; and the projection surface shape estimation unit241 that estimates a shape of the projection surface 104 on the basis ofthe corresponding point detected by the corresponding point detectionunit 239 may be included.

By configuring as described above, even in a case where the projectionimaging apparatus performs the processing related to the update of thecorrection information used for geometric correction and the like, theprocessing can be performed in a similar manner to the cases of thefirst embodiment and the second embodiment.

In addition, as in a projection imaging system 410 illustrated in B ofFIG. 21, the configurations of the respective projection imagingapparatuses may be different from each other. As illustrated in B ofFIG. 21, the projection imaging system 410 has a projection imagingapparatus 411, a projection imaging apparatus 412, and a projectionimaging apparatus 413 that are connected to a network 302 which is anarbitrary communication medium network so as to be able to communicatetherewith.

The projection imaging apparatus 411 has two projection units 111(projection units 111-1-1 and 111-1-2) and one imaging unit 112-1. Theprojection imaging apparatus 412 has one projection unit 111-2 and oneimaging unit 112-2. The projection imaging apparatus 413 has oneprojection unit 111-3 and two imaging units 112 (imaging units 112-3-1and 112-3-2).

Even in a case where the configurations of the respective projectionimaging apparatuses are different from each other as described above,the processing can be performed in a similar manner to the cases of thefirst embodiment and the second embodiment. That is, also in the case ofthis projection imaging system 410, an increase in processing amount ofthe processing related to the update of the correction information usedfor geometric correction and the like can be suppressed.

<Software>

A series of the above-described items of processing can be executed byhardware as well and also can be executed by software. In a case wherethe series of the above-described items of processing is executed bysoftware, a program constituting this software is installed from anetwork or a recording medium.

For example, in the case of the control apparatus 101 in FIG. 10, thisrecording medium is constituted by the removable medium 221 in which theprogram is recorded to be distributed to a user separately from theapparatus itself such that the program is delivered thereto. Thisremovable medium 221 includes a magnetic disk (including a flexibledisk) and an optical disc (including a CD-ROM and a DVD). Furthermore, amagneto-optical disk (including a mini disc (MD)) and a semiconductormemory are also included therein. In that case, for example, thisprogram stored in the removable medium 221 can be read and installed tothe storage unit 213 by mounting this removable medium 221 to the drive215.

In addition, for example, in the case of the projection imagingapparatus 102 in FIG. 12, this recording medium is constituted by theremovable medium 331 in which the program is recorded to be distributedto a user separately from the apparatus itself such that the program isdelivered thereto. This removable medium 331 includes a magnetic disk(including a flexible disk) and an optical disc (including a CD-ROM anda DVD). Furthermore, a magneto-optical disk (including an MD) and asemiconductor memory are also included therein. In that case, forexample, this program stored in the removable medium 331 can be read andinstalled to the storage unit 323 by mounting this removable medium 331to the drive 365.

In addition, this program can be also provided via a wired or wirelesstransmission medium such as a local area network, the Internet, ordigital satellite broadcasting. For example, in the case of the controlapparatus 101 in FIG. 10, the program can be received by thecommunication unit 214 to be installed to the storage unit 213.Meanwhile, for example, in the case of the projection imaging apparatus102 in FIG. 12, the program can be received by the communication unit324 to be installed to the storage unit 323.

As an alternative manner, this program also can be installed to astorage unit, a ROM, or the like in advance. For example, in the case ofthe control apparatus 101 in FIG. 30, the program also can be installedin advance to the storage unit 213, a ROM (not illustrated) built in thecontrol unit 201, or the like. Meanwhile, for example, in the case ofthe projection imaging apparatus 102 in FIG. 12, the program also can beinstalled in advance to the storage unit 323, a ROM (not illustrated)built in the control unit 311, or the like.

Note that, the program executed by a computer may be a program in whichthe items of processing are performed along the time series inaccordance with the order described in the present description, oralternatively, may be a program in which the items of processing areperformed in parallel or at a necessary timing, for example, whencalled.

In addition, in the present description, steps describing the program tobe recorded in a recording medium off course include processingperformed along the time series in accordance with the order in whichthe steps are described and additionally include processing notnecessarily processed along the time series but executed in a parallelmanner or individually.

Furthermore, the processing by the aforementioned respective steps canbe executed by the aforementioned respective apparatuses or anyapparatus other than the aforementioned respective apparatuses. In thatcase, this apparatus which is to execute the processing is simplyconfigured to have a function required to execute that aforementionedprocessing (e.g., a functional block). In addition, information requiredfor the processing can be configured to be transmitted to that apparatusas appropriate.

<Others>

Meanwhile, in the present description, the system refers to a collectionof a plurality of constituent members (e.g., apparatuses and modules(components)) and whether all the constituent members are arrangedwithin the same cabinet is not regarded as important. Therefore, aplurality of apparatuses accommodated in separate cabinets so as to beconnected to one another via a network and one apparatus of which aplurality of modules is accommodated within one cabinet are both deemedas systems.

Additionally, in the aforementioned cases, a configuration described asone apparatus (or a processing unit) may be divided so as to beconfigured as a plurality of apparatuses (or processing units). Orconversely, in the aforementioned cases, a configuration described as aplurality of apparatuses (or processing units) may be integrated so asto be configured as one apparatus (or a processing unit). In addition,as a matter of course, a configuration other than those described abovemay be employed to be added to the configurations of the respectiveapparatuses (or the respective processing units). Furthermore, a portionof a certain apparatus (or a certain processing unit) may be configuredto be included in the configuration of another apparatus (or anotherprocessing unit) as long as the configuration or the action of thesystem as a whole is maintained substantially unchanged.

The favorable embodiments of the present disclosure have been describedin detail thus far with reference to the accompanying drawings. However,the technological scope of the present disclosure is not limited tothese examples. It is clear that a person with average knowledge on thetechnological field of the present disclosure can arrive at variousvariations or modifications within a range of the technological spiritdisclosed in claims and as a matter of course, these are comprehended aspart of the technological scope of the present disclosure.

For example, the present technology can employ a cloud computingconfiguration in which one function is divided and allocated to aplurality of apparatuses so as to be processed in coordinationthereamong via a network.

Furthermore, the respective steps described in the aforementionedflowcharts can be executed by a plurality of apparatuses each taking ashare thereof as well as executed by a single apparatus.

Additionally, in a case where a plurality of items of processing isincluded in one step, the plurality of items of processing included inone step can be executed by a plurality of apparatuses each taking ashare thereof as well as executed by a single apparatus.

Meanwhile, the present technology is not limited thereto and can be alsocarried out as any configuration equipped in the apparatus describedabove or an apparatus constituting the system described above, forexample, a processor serving as system large scale integration (LSI) orthe like, a module using a plurality of processors or the like, a unitusing a plurality of modules or the like, a set in which anotherfunction is further added to a unit, or the like (that is, a partialconfiguration of an apparatus).

Note that the present technology can be also configured as describedbelow.

(1) An information processing apparatus including:

a corresponding point detection unit that detects, in regard toprojection imaging apparatuses each having a projection unit thatprojects an image on a projection surface and an imaging unit thatimages the projection surface, a corresponding point between a pixel ofthe projection unit and a pixel of the imaging unit for a relationbetween one of the projection imaging apparatuses whose posture haschanged and one of the projection imaging apparatuses whose posture hasnot changed; and

a relative posture estimation unit that estimates a relative posture ofthe projection imaging apparatus whose posture has changed with respectto the projection imaging apparatus whose posture has not changed on thebasis of the corresponding point between a pixel of the projection unitand a pixel of the imaging unit detected by the corresponding pointdetection unit for a relation between the projection imaging apparatuswhose posture has changed and the projection imaging apparatus whoseposture has not changed.

(2) The information processing apparatus according to (1), in which

the corresponding point detection unit detects a corresponding pointbetween a pixel of the projection unit of the projection imagingapparatus whose posture has not changed and a pixel of the imaging unitof the projection imaging apparatus whose posture has changed.

(3) The information processing apparatus according to (1) or (2), inwhich

the corresponding point detection unit detects a corresponding pointbetween a pixel of the projection unit of the projection imagingapparatus whose posture has changed and a pixel of the imaging unit ofthe projection imaging apparatus whose posture has not changed.

(4) The information processing apparatus according to any one of (1) to(3), further including:

a corresponding point change detection unit that detects a change in acorresponding point between a pixel of the projection unit and a pixelof the imaging unit of the projection imaging apparatuses;

a posture change detection unit that detects a posture change in eachprojection imaging apparatus on the basis of a sensor output from asensor unit which is included in each projection imaging apparatus anddetects at least one of a position and a direction; and

a change determination unit that determines whether a posture of eachprojection imaging apparatus has changed on the basis of a detectionresult of the corresponding point change detection unit and a detectionresult of the posture change detection unit, in which

the corresponding point detection unit is configured to detect thecorresponding point between one of the projection imaging apparatuseswhose posture is determined by the change determination unit to havechanged and one of the projection imaging apparatuses whose posture isdetermined by the change determination unit to have not changed, and

the relative posture estimation unit is configured to estimate arelative posture of the projection imaging apparatus whose posture isdetermined by the change determination unit to have changed with respectto the projection imaging apparatus whose posture is determined by thechange determination unit to have not changed.

(5) The information processing apparatus according to any one of (1) to(4), in which

the corresponding point detection unit is configured to further detect acorresponding point between a pixel of the projection unit and a pixelof the imaging unit of the projection imaging apparatus whose posturehas changed,

the information processing apparatus further including a projectionsurface shape estimation unit that estimates a shape of the projectionsurface on the basis of the corresponding point detected by thecorresponding point detection unit for a relation between a pixel of theprojection unit and a pixel of the imaging unit of the projectionimaging apparatus whose posture has changed.

(6) The information processing apparatus according to any one of (1) to(5), in which

the corresponding point detection unit is configured to further detect acorresponding point between a pixel of the projection unit and a pixelof the imaging unit of the projection imaging apparatus whose posturehas not changed,

the information processing apparatus further including a projectionsurface shape estimation unit that estimates a shape of the projectionsurface on the basis of the corresponding point detected by thecorresponding point detection unit for a relation between a pixel of theprojection unit and a pixel of the imaging unit of the projectionimaging apparatus whose posture has not changed.

(7) The information processing apparatus according to (5) or (6), inwhich

the projection surface shape estimation unit estimates a shape of theprojection surface for a part of the projection surface whose shape isunknown within a range on which an image is projected by the projectionunit of the projection imaging apparatus whose posture has changed.

(8) An information processing method configured to:

detect, in regard to projection imaging apparatuses each having aprojection unit that projects an image on a projection surface and animaging unit that images the projection surface, a corresponding pointbetween a pixel of the projection unit and a pixel of the imaging unitfor a relation between one of the projection imaging apparatuses whoseposture has changed and one of the projection imaging apparatuses whoseposture has not changed; and

estimate a relative posture of the projection imaging apparatus whoseposture has changed with respect to the projection imaging apparatuswhose posture has not changed on the basis of the corresponding pointbetween a pixel of the projection unit and a pixel of the imaging unitdetected for a relation between the projection imaging apparatus whoseposture has changed and the projection imaging apparatus whose posturehas not changed.

(9) An information processing apparatus including:

a corresponding point detection unit that detects, in regard to aprojection imaging apparatus having a projection unit that projects animage on a projection surface and an imaging unit that images theprojection surface, a corresponding point between a pixel of theprojection unit and a pixel of the imaging unit of the projectionimaging apparatus whose posture has not changed; and

a projection surface shape estimation unit that estimates a shape of theprojection surface on the basis of the corresponding point detected bythe corresponding point detection unit for a relation between a pixel ofthe projection unit and a pixel of the imaging unit of the projectionimaging apparatus whose posture has not changed.

(10) The information processing apparatus according to (9), in which

the projection surface shape estimation unit estimates a shape of theprojection surface for a part of the projection surface whose shape isunknown.

(11) An information processing method configured to:

detect, in regard to a projection imaging apparatus having a projectionunit that projects an image on a projection surface and an imaging unitthat images the projection surface, a corresponding point between apixel of the projection unit and a pixel of the imaging unit of theprojection imaging apparatus whose posture has not changed; and

estimate a shape of the projection surface on the basis of thecorresponding point detected for a relation between a pixel of theprojection unit and a pixel of the imaging unit of the projectionimaging apparatus whose posture has not changed.

(12) A projection imaging apparatus including:

a projection unit that projects an image on a projection surface;

an imaging unit that images the projection surface;

a sensor unit that detects at least one of a position and a direction;

a determination unit that determines whether a posture has changed onthe basis of a sensor output from the sensor unit;

a corresponding point detection unit that detects, in a case where thedetermination unit determines that the posture has changed, acorresponding point between a pixel of the projection unit and a pixelof the imaging unit for a relation with another projection imagingapparatus which includes the projection unit, the imaging unit, and thesensor unit and whose posture has not changed; and

a relative posture estimation unit that estimates a relative posturewith respect to the another projection imaging apparatus on the basis ofthe corresponding point detected by the corresponding point detectionunit.

(13) The projection imaging apparatus according to (12), in which

the corresponding point detection unit detects a corresponding pointbetween a pixel of the projection unit of the another projection imagingapparatus and a pixel of the own imaging unit of the projection imagingapparatus.

(14) The projection imaging apparatus according to (12) or (13), inwhich

the corresponding point detection unit detects a corresponding pointbetween a pixel of the own projection unit of the projection imagingapparatus and a pixel of the imaging unit of the another projectionimaging apparatus.

(15) The projection imaging apparatus according to any one of (12) to(14), further Including:

a corresponding point change detection unit that detects a change in acorresponding point between a pixel of the own projection unit and apixel of the own imaging unit of the projection imaging apparatus; and

a posture change detection unit that detects an own posture change inthe projection imaging apparatus on the basis of a sensor output fromthe sensor unit, in which

the determination unit is configured to determine whether an own postureof the projection imaging apparatus has changed on the basis of adetection result of the corresponding point change detection unit and adetection result of the posture change detection unit.

(16) The projection imaging apparatus according to any one of (12) to(15), in which

the corresponding point detection unit is configured to further detect acorresponding point between a pixel of the own projection unit and apixel of the own imaging unit of the projection imaging apparatus,

the projection imaging apparatus further including a projection surfaceshape estimation unit that estimates a shape of the projection surfaceon the basis of the corresponding point detected by the correspondingpoint detection unit for a relation between a pixel of the ownprojection unit and a pixel of the own imaging unit of the projectionimaging apparatus.

(17) The projection imaging apparatus according to (16), in which

the projection surface shape estimation unit estimates a shape of theprojection surface for a part of the projection surface whose shape isunknown within a range on which an image is projected by the ownprojection unit of the projection imaging apparatus.

(18) An information processing method for a projection imaging apparatusincluding a projection unit that projects an image on a projectionsurface; an imaging unit that images the projection surface; and asensor unit that detects at least one of a position and a direction,

the information processing method being configured to:

determine whether a posture has changed on the basis of a sensor outputfrom the sensor unit;

detect, in a case where the posture is determined to have changed, acorresponding point between a pixel of the projection unit and a pixelof the imaging unit for a relation with another projection imagingapparatus which includes the projection unit, the imaging unit, and thesensor unit and whose posture has not changed; and

estimate a relative posture with respect to the another projectionimaging apparatus on the basis of the detected corresponding point.

(19) A projection imaging apparatus including:

a projection unit that projects an image on a projection surface;

an imaging unit that images the projection surface;

a sensor unit that detects at least one of a position and a direction;

a determination unit that determines whether a posture has changed onthe basis of a sensor output from the sensor unit;

a corresponding point detection unit that detects, in a case where thedetermination unit determines that the posture has not changed, acorresponding point between a pixel of the own projection unit and apixel of the own imaging unit of the projection imaging apparatus; and

a projection surface shape estimation unit that estimates a shape of theprojection surface on the basis of the corresponding point detected bythe corresponding point detection unit.

(20) An information processing method for a projection imaging apparatusincluding a projection unit that projects an image on a projectionsurface; an imaging unit that images the projection surface; and asensor unit that detects at least one of a position and a direction,

the information processing method being configured to:

determine whether a posture has changed on the basis of a sensor outputfrom the sensor unit;

detect, in a case where the posture is determined to have not changed, acorresponding point between a pixel of the own projection unit and apixel of the own imaging unit of the projection imaging apparatus; and

estimate a shape of the projection surface on the basis of the detectedcorresponding point.

REFERENCE SIGNS LIST

-   100 Projection imaging system-   101 Control apparatus-   102 Projection imaging apparatus-   103 Communication cable-   104 Projection surface-   105 Projection image-   106 Input image-   107 Corrected projection image-   111 Projection unit-   112 Imaging unit-   121 Three-dimensional object-   201 Control unit-   231 Processing control unit-   232 Projection control unit-   233 Imaging control unit-   234 Sensor control unit-   235 Image projection unit-   236 Corresponding point change detection unit-   237 Sensor information change detection unit-   238 Change determination unit-   239 Corresponding point detection unit-   240 Posture estimation unit-   241 Projection surface shape estimation unit-   242 Correction information update unit-   301 Projection port-   302 Imaging port-   303 Battery-   311 Control unit-   312 Sensor unit-   400 Projection imaging system-   410 Projection imaging system-   411 Projection imaging apparatus-   412 Projection imaging apparatus-   413 Projection imaging apparatus-   414 Network

The invention claimed is:
 1. An information processing apparatuscomprising: a corresponding point detection unit that detects, in regardto projection imaging apparatuses each having a projection unit thatprojects an image on a projection surface and an imaging unit thatimages the projection surface, a corresponding point between a pixel ofthe projection unit and a pixel of the imaging unit for a relationbetween one of the projection imaging apparatuses whose posture haschanged and one of the projection imaging apparatuses whose posture hasnot changed; and a relative posture estimation unit that estimates arelative posture of the projection imaging apparatus whose posture haschanged with respect to the projection imaging apparatus whose posturehas not changed on the basis of the corresponding point between a pixelof the projection unit and a pixel of the imaging unit detected by thecorresponding point detection unit for a relation between the projectionimaging apparatus whose posture has changed and the projection imagingapparatus whose posture has not changed, wherein the corresponding pointdetection unit and the relative posture estimation unit are eachimplemented via at least one processor.
 2. The information processingapparatus according to claim 1, wherein the corresponding pointdetection unit detects a corresponding point between a pixel of theprojection unit of the projection imaging apparatus whose posture hasnot changed and a pixel of the imaging unit of the projection imagingapparatus whose posture has changed.
 3. The information processingapparatus according to claim 1, wherein the corresponding pointdetection unit detects a corresponding point between a pixel of theprojection unit of the projection imaging apparatus whose posture haschanged and a pixel of the imaging unit of the projection imagingapparatus whose posture has not changed.
 4. The information processingapparatus according to claim 1, further comprising: a correspondingpoint change detection unit that detects a change in a correspondingpoint between a pixel of the projection unit and a pixel of the imagingunit of the projection imaging apparatuses; a posture change detectionunit that detects a posture change in each projection imaging apparatuson the basis of a sensor output from a sensor unit which is included ineach projection imaging apparatus and detects at least one of a positionand a direction; and a change determination unit that determines whethera posture of each projection imaging apparatus has changed on the basisof a detection result of the corresponding point change detection unitand a detection result of the posture change detection unit, wherein thecorresponding point detection unit is configured to detect thecorresponding point between one of the projection imaging apparatuseswhose posture is determined by the change determination unit to havechanged and one of the projection imaging apparatuses whose posture isdetermined by the change determination unit to have not changed, therelative posture estimation unit is configured to estimate a relativeposture of the projection imaging apparatus whose posture is determinedby the change determination unit to have changed with respect to theprojection imaging apparatus whose posture is determined by the changedetermination unit to have not changed, and the corresponding pointchange detection unit, the posture change detection unit, and the changedetermination unit are each implemented via at least one processor. 5.The information processing apparatus according to claim 1, wherein thecorresponding point detection unit is configured to further detect acorresponding point between a pixel of the projection unit and a pixelof the imaging unit of the projection imaging apparatus whose posturehas changed, and the information processing apparatus further comprisinga projection surface shape estimation unit that estimates a shape of theprojection surface on the basis of the corresponding point detected bythe corresponding point detection unit for a relation between a pixel ofthe projection unit and a pixel of the imaging unit of the projectionimaging apparatus whose posture has changed.
 6. The informationprocessing apparatus according to claim 5, wherein the projectionsurface shape estimation unit estimates a shape of the projectionsurface for a part of the projection surface whose shape is unknownwithin a range on which an image is projected by the projection unit ofthe projection imaging apparatus whose posture has changed.
 7. Theinformation processing apparatus according to claim 1, wherein thecorresponding point detection unit is configured to further detect acorresponding point between a pixel of the projection unit and a pixelof the imaging unit of the projection imaging apparatus whose posturehas not changed, and the information processing apparatus furthercomprising a projection surface shape estimation unit that estimates ashape of the projection surface on the basis of the corresponding pointdetected by the corresponding point detection unit for a relationbetween a pixel of the projection unit and a pixel of the imaging unitof the projection imaging apparatus whose posture has not changed. 8.The information processing apparatus according to claim 7, wherein theprojection surface shape estimation unit estimates a shape of theprojection surface for a part of the projection surface whose shape isunknown within a range on which an image is projected by the projectionunit of the projection imaging apparatus whose posture has changed. 9.An information processing method configured to: detect, in regard toprojection imaging apparatuses each having a projection unit thatprojects an image on a projection surface and an imaging unit thatimages the projection surface, a corresponding point between a pixel ofthe projection unit and a pixel of the imaging unit for a relationbetween one of the projection imaging apparatuses whose posture haschanged and one of the projection imaging apparatuses whose posture hasnot changed; and estimate a relative posture of the projection imagingapparatus whose posture has changed with respect to the projectionimaging apparatus whose posture has not changed on the basis of thecorresponding point between a pixel of the projection unit and a pixelof the imaging unit detected for a relation between the projectionimaging apparatus whose posture has changed and the projection imagingapparatus whose posture has not changed.
 10. An information processingapparatus comprising: a corresponding point detection unit that detects,in regard to a projection imaging apparatus having a projection unitthat projects an image on a projection surface and an imaging unit thatimages the projection surface, a corresponding point between a pixel ofthe projection unit and a pixel of the imaging unit of the projectionimaging apparatus whose posture has not changed during the projection ofthe image; and a projection surface shape estimation unit that estimatesa shape of only a portion of the projection surface that extends outsideof an initial projection range of the projection surface, the portion ofthe projection surface being smaller than a total surface of theprojection surface and whose shape is unknown within a range on whichthe image is projected, by performing depth sensing on the basis of thecorresponding point detected by the corresponding point detection unitfor a relation between a pixel of the projection unit and a pixel of theimaging unit of the projection imaging apparatus whose posture has notchanged during the projection of the image, wherein the correspondingpoint detection unit and the projection surface shape estimation unitare each implemented via at least one processor.
 11. The informationprocessing apparatus according to claim 10, wherein the projectionsurface shape estimation unit estimates the shape of the portion of theprojection surface based on correspondence relationship of pixelsbetween the projection unit and the imaging unit.
 12. The informationprocessing apparatus according to claim 11, wherein the correspondencerelationship is expressed as which pixel of the imaging unit is a pixelto which a certain pixel of the projection unit corresponds.
 13. Theinformation processing apparatus according to claim 10, wherein theinitial projection range of the projection surface comprises thoseportions of the projection surface upon which one or more images havebeen projected thereon from a plurality of projection imagingapparatuses prior to a change in posture of one projection imagingapparatus of the plurality of projection imaging apparatuses.
 14. Theinformation processing apparatus according to claim 13, wherein theportion of the projection surface that extends outside of the initialprojection range of the projection surface corresponds to a portion ofthe projection surface upon which one or more images have been projectedthereon from the plurality of projection imaging apparatuses only afterthe change in posture of the one projection imaging apparatus of theplurality of projection imaging apparatuses.
 15. The informationprocessing apparatus according to claim 14, wherein, prior to the changein posture of the one projection imaging apparatus of the plurality ofprojection imaging apparatuses, the portion of the projection surfacethat extends outside of the initial projection range of the projectionsurface does not have projected thereon any images from the plurality ofimaging apparatus.
 16. An information processing method configured to:detect, in regard to a projection imaging apparatus having a projectionunit that projects an image on a projection surface and an imaging unitthat images the projection surface, a corresponding point between apixel of the projection unit and a pixel of the imaging unit of theprojection imaging apparatus whose posture has not changed during theprojection of the image; and estimate a shape of only a portion of theprojection surface that extends outside of an initial projection rangeof the projection surface, the portion of the projection surface beingsmaller than a total surface of the projection surface and whose shapeis unknown within a range on which the image is projected, by performingdepth sensing on the basis of the corresponding point detected for arelation between a pixel of the projection unit and a pixel of theimaging unit of the projection imaging apparatus whose posture has notchanged during the projection of the image.
 17. A projection imagingapparatus comprising: a projection unit that projects an image on aprojection surface; an imaging unit that images the projection surface;a sensor unit that detects at least one of a position and a direction; adetermination unit that determines whether a posture has changed on thebasis of a sensor output from the sensor unit; a corresponding pointdetection unit that detects, in a case where the determination unitdetermines that the posture has changed, a corresponding point between apixel of the projection unit and a pixel of the imaging unit for arelation with another projection imaging apparatus which includes theprojection unit, the imaging unit, and the sensor unit and whose posturehas not changed; and a relative posture estimation unit that estimates arelative posture with respect to the another projection imagingapparatus on the basis of the corresponding point detected by thecorresponding point detection unit, wherein the projection unit, theimaging unit, the sensor unit, the determination unit, the correspondingpoint detection unit, and the relative posture estimation unit are eachimplemented via at least one processor.
 18. The projection imagingapparatus according to claim 17, wherein the corresponding pointdetection unit detects a corresponding point between a pixel of theprojection unit of the another projection imaging apparatus and a pixelof the own imaging unit of the projection imaging apparatus.
 19. Theprojection imaging apparatus according to claim 17, wherein thecorresponding point detection unit detects a corresponding point betweena pixel of the own projection unit of the projection imaging apparatusand a pixel of the imaging unit of the another projection imagingapparatus.
 20. The projection imaging apparatus according to claim 17,further comprising: a corresponding point change detection unit thatdetects a change in a corresponding point between a pixel of the ownprojection unit and a pixel of the own imaging unit of the projectionimaging apparatus; and a posture change detection unit that detects anown posture change in the projection imaging apparatus on the basis of asensor output from the sensor unit, wherein the determination unit isconfigured to determine whether an own posture of the projection imagingapparatus has changed on the basis of a detection result of thecorresponding point change detection unit and a detection result of theposture change detection unit, and the corresponding point changedetection unit and the posture change detection unit are eachimplemented via at least one processor.
 21. The projection imagingapparatus according to claim 17, wherein the corresponding pointdetection unit is configured to further detect a corresponding pointbetween a pixel of the own projection unit and a pixel of the ownimaging unit of the projection imaging apparatus, and the projectionimaging apparatus further comprising a projection surface shapeestimation unit that estimates a shape of the projection surface on thebasis of the corresponding point detected by the corresponding pointdetection unit for a relation between a pixel of the own projection unitand a pixel of the own imaging unit of the projection imaging apparatus.22. The projection imaging apparatus according to claim 21, wherein theprojection surface shape estimation unit estimates a shape of theprojection surface for a part of the projection surface whose shape isunknown within a range on which an image is projected by the ownprojection unit of the projection imaging apparatus.
 23. An informationprocessing method for a projection imaging apparatus comprising aprojection unit that projects an image on a projection surface; animaging unit that images the projection surface; and a sensor unit thatdetects at least one of a position and a direction, the informationprocessing method being configured to: determine whether a posture haschanged on the basis of a sensor output from the sensor unit; detect, ina case where the posture is determined to have changed, a correspondingpoint between a pixel of the projection unit and a pixel of the imagingunit for a relation with another projection imaging apparatus whichcomprises the projection unit, the imaging unit, and the sensor unit andwhose posture has not changed; and estimate a relative posture withrespect to the another projection imaging apparatus on the basis of thedetected corresponding point.
 24. A projection imaging apparatuscomprising: a projection unit that projects an image on a projectionsurface; an imaging unit that images the projection surface; a sensorunit that detects at least one of a position and a direction; adetermination unit that determines whether a posture has changed duringthe projection of the image on the basis of a sensor output from thesensor unit; a corresponding point detection unit that detects, in acase where the determination unit determines that the posture has notchanged during the projection of the image, a corresponding pointbetween a pixel of the own projection unit and a pixel of the ownimaging unit of the projection imaging apparatus; and a projectionsurface shape estimation unit that estimates a shape of only a portionof the projection surface that extends outside of an initial projectionrange of the projection surface, the portion of the projection surfacebeing smaller than a total surface of the projection surface and whoseshape is unknown within a range on which the image is projected, byperforming depth sensing on the basis of the corresponding pointdetected by the corresponding point detection unit, wherein theprojection unit, the imaging unit, the sensor unit, the determinationunit, the corresponding point detection unit, and the projection surfaceshape estimation unit are each implemented via at least one processor.25. An information processing method for a projection imaging apparatuscomprising a projection unit that projects an image on a projectionsurface; an imaging unit that images the projection surface; and asensor unit that detects at least one of a position and a direction, theinformation processing method being configured to: determine whether aposture has changed during the projection of the image on the basis of asensor output from the sensor unit; detect, in a case where the postureis determined to have not changed during the projection of the image, acorresponding point between a pixel of the own projection unit and apixel of the own imaging unit of the projection imaging apparatus; andestimate a shape of only a portion of the projection surface thatextends outside of an initial projection range of the projectionsurface, the portion of the projection surface being smaller than atotal surface of the projection surface and whose shape is unknownwithin a range on which the image is projected, by performing depthsensing on the basis of the detected corresponding point.